Methods of the object stored in hermione.ctx are not passed to the test about testplane

[shadowusr]

Hey!

You're absolutely right, object methods cannot be serialized/deserialized when passing between processes and therefore are lost.

We'll make it more clear in the docs.

Can you elaborate on your use-case, perhaps you need to use another feature of hermione instead of passing objects through hermione.ctx.

[shadowusr]

Thanks for providing the details, @vitaly-kiselev-qa.

I understood your use case, but it's not entirely clear to me what you'd like to achieve in the end. Why would you want to check whether the test is muted while running the test?

Generally things like tinkering with test's status (skipping or muting test) should be done in the master process and tests themselves shouldn't be aware of such machinery. Understanding your end goal would help me suggest a better workaround.

[shadowusr]

You can achieve described behaviour by intercepting hermione events in the master process. Here's a short example:

hermione.intercept(hermione.events.TEST_FAIL, ({event, data: test}) => {
     if (externalDataSource.shouldMute(test)) {
        return {event: hermione.events.TEST_PENDING, data: test};
    }
 
    return {event, data: test};
});

Does this solution satisfy your requirements?

[shadowusr]

@vitaly-kiselev-qa, hmm, maybe I'm misunderstanding something, but I still see no issue in doing that inside event interceptor in master process. This may look like this:

hermione.intercept(hermione.events.TEST_FAIL, ({event, data: test}) => {
     if (externalDataSource.shouldMute(test)) {
        test.skipReason = "Muted due to " + externalDataSource.getMuteReason(test);

       // Or you may save any info to meta and use it later in your report:
       test.meta.muteReason = { reason: externalDataSource.getMuteReason(test), details: /* ... */ };

        return {event: hermione.events.TEST_PENDING, data: test};
    }
 
    return {event, data: test};
});

Hope I got your intentions right, if you have more questions left, I'm ready to answer =)

[shadowusr]

I'm glad that worked. Feel free to open a new issue if you need help in the future!

[vitaly-kiselev-qa]

@shadowusr
Thank you for looking at the issue.

I'll try to describe our use-case.

• Our muted autotest list is stored in an external database.
• In the BEGIN event handler the list is get from the database.
• In the TEST_FAIL event handler there is a check if the failed test is muted.

Now I see, that the list itself can be passed between BEGIN and TEST_FAIL event handlers.
But functions which process this data (for example, check whether test muted or not) — can not.

Am I correct?

Is there any other Hermione feature I can use for this purpose?
Or I should just store functions somethere else and continue using hermione.ctx to transfer the data?

[vitaly-kiselev-qa]

@shadowusr

Sorry for being not clear.

Our goal is to skip failed tests if they are muted.
In the TEST_FAIL event handler we check whether the test muted or not. If it is muted we change the status to SKIPPED.

As a result we have the following behavior:

• a muted test passes — it has PASSED status;
• a muted test fails — it has SKIPPED status. In this case we re-check the case manually.

We do not want to skip all muted tests to reduce amount of manual testing.
We do not want to have them failed because they break our CI pipelines and our managed don't like broken pipelines :-)

[vitaly-kiselev-qa]

@shadowusr
Thank you for the example.
Looks like we already use similar code taken from the Hermione documentation.

I realized that I have misleaded you. Sorry for that.

There is one more scenario we want to implement: to have information from externalDataSource in the test report.
The goal is to see something like 'The test is muted because of the issue <...>' in the test report.
That's why I asked about accessing externalDataSource in the test itself.

Again, sorry for not being clear.

[vitaly-kiselev-qa]

@shadowusr
You got it right!

But there is an issue.
Our currect Allure report plugin https://github.com/allure-framework/allure-js/tree/main/packages/allure-hermione does not pass neigher skipReason no meta to the report.
That's why we had to do it on our side via test links.

Added after a second thought
Maybe we can just put meta to the report in the end of each test 🤔
Thank you very much for the idea!