[🐛 Bug]: WebdriverIO toMatchSnapshot does not fail tests on mocha test retries about webdriverio

[BorisOsipov]

@Winify There is only one strange thing that I suspect -
should adequately assert texts (1 retries)
(1 retries) - What kind of retries do you use? could you check with and without retries?

[BorisOsipov]

@Winify We need a repository with a reproducible example. Your example and repro steps work well for me.

[0-0] Error in "Snapshot testing.should adequately assert texts"
Error: expect(received).toBe(expected) // Object.is equality

- Expected  - 0
+ Received  + 1

  "
....
    Broken Images
+   Challenging DOM
    Checkboxes
    Context Menu
    Digest Authentication (user and pass: admin)
....
  "
    at Context.<anonymous>
[0-0] FAILED in chrome 

Snapshot Summary:
1 snapshot(s) unmatched.

Spec Files:      0 passed, 1 failed, 1 total (100% completed) in 00:00:05

[wdio-bot]

Thanks for raising this issue 🙏

Unfortunately we can't help you without a reproducible example in this matter. Please read our contributing guidelines on how to create a reproducible example. If you can't provide a reproducible example we will close this issue in 7 days.

If you have any questions please reach out to us on our Discord channel. We are happy to help you out there.

[Winify]

@BorisOsipov Thank you for the swift response,

And I apologize, because as I created a repro example project.... it actually works as expected.
It looks like something in our framework goes awry, but I could not identify what, as I copied our configs to the repro project as well...

All in all, thank you so much

[christian-bromann]

Thanks @Winify , let us know if you are able to reproduce this one, then I am happy to re-open and take a look.

[Winify]

@BorisOsipov You are right!

I am using mocha retries:

  mochaOpts: {
    ui: 'bdd',
    timeout: process.env.SPEC_TIMEOUT ?? 90_000,
    compilers: ['ts:ts-node/register'],
    retries: Number(process.env.RETRIES ?? 0)
  },

In the repro project I emitted this config, as I did not think it was neccessary.

If I remove the RETRIES=1 config from the env file, it fails as expected!

[wdio-bot]

Thanks for reporting!

We greatly appreciate any contributions that help resolve the bug. While we understand that active contributors have their own priorities, we kindly request your assistance if you rely on this bug being fixed. We encourage you to take a look at our contribution guidelines or join our friendly Discord development server, where you can ask any questions you may have. Thank you for your support, and cheers!