pvlakshm / mstestex Goto Github PK

Description

A test methos set to retry in set number of times shown as failed in VS2017 test runner.

Steps to reproduce

setup a test in VS2017. Set method to retry 5 times.

Expected behavior

If test failed first attempt it should retry to execute the test and on second attempt if test run successfully test runner should show array of test results and test ruunner highlight test as successfull.

Actual behavior

Test runner show test as failed.

Environment

v1.0.2

How i have overcome as of now.

on success of a retry, atpreset im filtering failed testresult and returning only success trest result from arrat of testresult.

Please add the ability to your MSTestEx not to fail the test if it failed on the first try but passed on second or third try (depending on settings)

From @SturlaThorvalds here.

Description

Given that I run test with retry attribute with the value of N and some of those times test fails, mixed results are returned to MSTest framework, and as the result test is marked as failed.

Steps to reproduce

Create test with retry attribute, with the value of 5
Fail test 4 times out of 5

Expected behavior

Test run is marked as successful

Actual behavior

Test run is not marked as successful

Description

I would want to caught and made available the exception thrown by Test Method in Test Cleanup and update to TFS or any other test management tool under the Test case associated with Test Method.

Steps to reproduce

What steps can reproduce the defect?
Please share the setup, commandline for vstest.console, sample project, target
framework etc.

Expected behavior

Share the expected output

Actual behavior

What is the behavior observed?

Environment

Please share additional details about the test environment.
Operating system, Build version of vstest.console

From @dotMorten
... create a new TestMethodExAttribute instead and make the retry an attribute. Makes it possible to create all sorts of attributes that control execution.