Asynchronous tests concurrency problems. about respawn

[jbogard]

Other ways you can address this is with Dev Containers or separate Local DB instances. But using this library in those cases has diminishing returns of course.

[Insire]

I've a similar use case for Respawn, although the setup seems to be a bit more involved than yours @sergey-bulavskiy .

I create an inmemory pool of databases, that i also manage in a separate ms sql server db(TestManagementDb). Each test run creates its own set of up to 16 databases. Each time, a test needs a database, i ask my pool for an instance. Each instance knows if there is a corresponding db already created(raw sql), whether migrations(EF Core) have been run on it and whether it has been seeded with default data(EF Core).

If anything of that is still missing, i'll fix that before returning the database as EF Core dbcontext to my test. Then i run my test logic as usual and then dispose it via IAsyncDisposable, which in turn uses Respawn to clean up the data in the database and also update its state in my TestManagementDb until finally i return the database back to the pool.

Then the next test can fetch an instance from the pool. I also use Semaphore slim here.

This approach is incredibly fast, provided you can figure out the locking around the pool and how to deal with failing and cancelled tests. In my case, when all tests are done, i drop all of my 16 databases and remove the corresponding entries from my TestManagementDb aswell.

[Insire]

Hello, @Insire! Thanks for you experience, can you please clarify a couple of moments? What test framework are you using? Are your tests all in one project? I was trying to achieve concurrency via semaphore slim and database pool via xunit fixtures but failed miserably because of xunit tricky parallel execution of the tests, and xunit fixtures being project-wide at maximum. To correctly support pool for multiple projects we'd need to use Assembly-level fixture which is not supported yet.

I am using x-unit with https://github.com/JDCain/Xunit.Extensions.AssemblyFixture which does add the assembly level fixture you mentioned.

I have multple projects

• unit tests (no database involved here)
• integration tests (the majority of our tests use ms sql server as database here)
• performance tests (a form of integration test)
• smoke tests (a form of integration test)
• bdd tests (a form of integration test)
• ui tests (a form of integration test)
• bdd-ui tests (a form of integration test)

[keithn]

you can stop xunit running things in parallel by putting a collection attribute on the test class [Collection("Anything with this name won't be run in parallel")]

[sergey-bulavskiy]

@keithn so it is intended way to use Respawn, only with sequential tests execution?

[keithn]

I can't see how you'd expect tests against a single state store to work?

[sergey-bulavskiy]

@keithn indeed, i thought that somebody faced same problems previously and managed it somehow, but i'll stick to sequential runs, thank you.

[sergey-bulavskiy]

Hello, @Insire!
Thanks for you experience, can you please clarify a couple of moments?
What test framework are you using? Are your tests all in one project?
I was trying to achieve concurrency via semaphore slim and database pool via xunit fixtures but failed miserably because of xunit tricky parallel execution of the tests, and xunit fixtures being project-wide at maximum. To correctly support pool for multiple projects we'd need to use Assembly-level fixture which is not supported yet.

[sergey-bulavskiy]

Thank you very much, @Insire, will have a look on this assembly fixture!