Improved location of extensions about nunit-console

[davidjward30]

I'm running nunit under teamcity and everything worked fine until we ended up with 2 versions of nunit3 in the repository (3.2.1 and 3.4.1). This tends to happen as different product teams migrate at different times. This caused the nunit 3.2.1 tests to immediately fail:

ProjectLoader extension .nunit is already handled by another extension.

The extensions are checked in with the standard nuget layout

[CharliePoole]

@davidjward30 Your problem isn't really related to this issue, which is about how the engine actually finds extensions. It's definitely intentional that you can't install the same extension twice, as you might imagine. The code checks its own behavior, eliminating any duplication in the actual files it examines but if you have two different copies of an extension in different directories, it will try to use both of them.

Extension installation requires manual tuning if you do anything except use the default install. There is no reason why you can't use different versions of NUnit in different projects, because the nunit package is just the framework and has nothing to do with extensions. Having multiple versions of the engine and of extensions is another matter. You should never do that within a solution.

It would require much more detail about what you are doing to give any specific suggestions for how to change your setup. If you need that - if the above doesn't tell you what to do - please ask about it in then nunit-discuss list rather than continuing here.

[rprouse]

@davidjward30 to elaborate on what @CharliePoole said, it is fine to use two different versions of the NUnit framework in your solutions, but you should try to use the latest version of NUnit Console. Multiple versions should work, but if I remember right, we had a bug in 3.2 where it finds all extensions including duplicates. That has been fixed in 3.4.

Just update your Console runner to 3.4.1+ and use it to run your 3.2.1 tests.

[davidjward30]

@CharliePoole @rprouse Thanks for the advice; indeed I can run 3.4 and 3.2 tests with the 3.4.1 runner.

[CharliePoole]

Situation brought up in an email discussion by @NikolayPianikov ...

Just run these commands one by one:

nuget.exe install NUnit.Runners -Version 3.4.1 -o packages

nuget.exe install NUnit.Runners -Version 3.5 -o packages

nuget.exe install NUnit.Runners -Version 3.6 -o packages

After that run a NUnit console from one of directories packages\NUnit.ConsoleRunner.x.x.x

It is a general case I think. For simplicity, let's assume that they are backwards compatible J The question is which extensions will be used? Do we have any rules on that?

[sergiorykov]

Do you have strategy to choose compatible versions of extensions for runner?

nunit.nuget.addins:

../../NUnit.Extension.*/**/tools/     # nuget v2 layout
../../../NUnit.Extension.*/**/tools/  # nuget v3 layout

Eventually, after a couple of updates (without cleaning packages folder) you will get several versions of the same packages.

14.09.2016  18:44    <DIR>          NUnit.Console.3.0.1
21.09.2016  11:55    <DIR>          NUnit.Console.3.4.1
12.10.2016  15:42    <DIR>          NUnit.Console.3.5.0
21.09.2016  11:55    <DIR>          NUnit.ConsoleRunner.3.4.1
12.10.2016  15:42    <DIR>          NUnit.ConsoleRunner.3.5.0
21.09.2016  11:55    <DIR>          NUnit.Extension.NUnitProjectLoader.3.4.1
12.10.2016  15:42    <DIR>          NUnit.Extension.NUnitProjectLoader.3.5.0
21.09.2016  11:55    <DIR>          NUnit.Extension.NUnitV2Driver.3.4.1
12.10.2016  15:42    <DIR>          NUnit.Extension.NUnitV2Driver.3.5.0
21.09.2016  11:55    <DIR>          NUnit.Extension.NUnitV2ResultWriter.3.4.1
12.10.2016  15:42    <DIR>          NUnit.Extension.NUnitV2ResultWriter.3.5.0
21.09.2016  11:55    <DIR>          NUnit.Extension.TeamCityEventListener.1.0.1
12.10.2016  15:42    <DIR>          NUnit.Extension.TeamCityEventListener.1.0.2
21.09.2016  11:55    <DIR>          NUnit.Extension.VSProjectLoader.3.4.1
12.10.2016  15:42    <DIR>          NUnit.Extension.VSProjectLoader.3.5.0

[sergiorykov]

As for me to get clear understanding of migration process and how extensibility works will be enough:

1. in NUnit.Console add `tools\readme.txt'

• with suggestion to move path to NUnit.ConsoleRunner\tools
• link to github readme about extensibility and migration
• warning that you must add all dependent packages to packages.config if it was independent installation using command line nuget.exe

2. improve Readme.md

• migration plan from 3.0.1 and earlier (must be there - you broke backward compability in a minor release (3.0 -> 3.5))
• how extensibility works (all that magic)
• section about independent installation using 'nuget.exe' (user should explicitly specify all required packages) - it's limitation of nuget.exe - it willn't download all dependent packages. it's one of it's serious drawbacks

Hope this helps!

It's a point of view of a man who haven't seen all this functionality from the ground up as you saw it :)

[rprouse]

Eventually, after a couple of updates (without cleaning packages folder) you will get several versions of the same packages.

As of NUnit 3.2.1 (I think that is the version 😄 ), the engine automatically uses the latest version of each extension it finds and ignores older versions.

[CharliePoole]

@sergiorykov As Rob points out, we choose the "best" (newest) version of each extension package. There isn't a concept of compatibility wrt a particular engine version because extensions are required to be backward compatible. They can only specify a minimum engine version under which they work.

So with the directory example you posted, the engine would use version 1.0.2 of the teamcity extension and 3.5.0 of all the other extensions. The console / engine version would depend on which of the two available you ran.

We should have a test that verifies this, however.

I still have to look at the original 3.0 layout of files. If, as you say, all the files were actually contained in the package rather than handled as dependencies, we should document the change. I'm not sure, however, if the info really belongs in Readme.md, which seems to be getting a lot of work to do. :-) We do have pages and pages of docs where this could go, including the Breaking Changes page.

I'm still confused by the notion that nuget.exe won't download the dependent packages. I'm pretty sure that is what it does in our CI builds. Can you post a sample command line that demonstrates the problem?

[garuma]

Just to add another case to this issue: I would like to provide a framework driver extension to be found by Visual Studio NUnit test provider which, due to the current path limitations, means that a custom .addin file needs to be placed in the same directory than the engine DLL as it's the only thing I found would work from the current 3.6 (https://github.com/nunit/nunit-console/blob/release/3.6/src/NUnitEngine/nunit.engine/Services/ExtensionService.cs#L183).

This makes it quite impractical to distribute said extension since it requires manual copying from the user or re-packaging the original NUnit provider NuGet to include the extra extension.