jwosty / monodevelop.unittests.expecto

Expecto test adapter for MonoDevelop / Visual Studio for Mac

MonoDevelop.UnitTests.Expecto

A MonoDevelop / Visual Studio for Mac addin that provides an Expecto test adapter

This project is in its early stages. Contributions are always welcome.

Test discovery

The addin currently discovers tests by searching the user assemblies for TestsAttribute, as in:

[<Tests>]
let test1 = test "hello world" { () }

Building and running

Open MonoDevelop.UnitTests.Expecto.sln and run the MonoDevelop.UnitTests.Expecto project from Visual Studio for Mac / MonoDevelop.

Installing

Run:

msbuild MonoDevelop.UnitTesting.Expecto/MonoDevelop.UnitTesting.Expecto.fsproj /p:Configuration=Release /p:InstallAddin=true

NOTE: This has project has not been tested on Windows MonoDevelop. It probably doesn't work (but it's probably not too much effort to make it work)

monodevelop.unittests.expecto's Issues

Make stdout from test runs show up in their output pad

This is a bit of a tricky one. It would be certainly doable for sequenced tests by injecting our own custom console stream that changes per test, but I'm not sure it's possible at all for asynchronous tests -- since asynchronous tests are switching threads the time, how do we know which test is performing any given call to Console.Write??

Test extension leaks test runner processes

Cracking open Activity Monitor reveals a bunch of rogue mono-sgen64 processes, many of which are instances of the external test runner, even after the IDE has been closed.

Workaround

Cull them by running killall mono from time to time.

Support running tests under .Net Core and other platforms

Also, other .Net framework versions.

This should be pretty easy. Steps required:

1. Create stub test runner server host projects for each platform, using Expecto.RunnerServer.Net461.fsproj as a basis, and add each as a dependency to the main addin
2. Go into the code for RemoteTestRunner (inside TestDiscoverer.fs) and add logic to the launching code to launch a different exe based on which platform the user wants to run under (either that or based off of the target framework of the user's project being built)

Support detecting tests without TestAttribute

Don't show projects that don't have tests in the tests pane

Respect test run modes (parallel vs sequential)

Currently, the extension always runs tests in parallel, not respecting if they're configured to run sequentially.

Implement "show test source code" and "go to failure"

Tests with '/' as part of the name aren't handled correctly

E.g.

This happens because the extension gets a flat list of tests (names concatenated with slashes) from the external test runner, and therefore can't tell the different between a real slash that was inserted to indicate hierarchy, and a slash that's actually just part of the tests's name. See: /MonoDevelop.UnitTesting.Expecto/ExpectoTestSuites.fs#L251

Possible solutions:

• Escape slashes in test names in the external test runner (this feels a little dirty but would work)
• Send the tests back to the client in a tree form (can't use the actual Expecto.Test since this isn't serializable, so we'd have to define our own Test tree type that mirrors it, but without the Expecto.TestCode)
• Change FlatTestInfo.name to be name: string list instead of name: string, where each element is the hierarchy. (Of course code will have to be changed in the test runner to create the correct structure)