I have installed the package to my test project, ran dotnet test /p:CollectCoverage=true as well as dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=opencover, but no code coverage report was generated.

dotnet SDK: 1.0.4
Taregt Framework: netcoreapp1.1
xunit: 2.3.1
coverlet.msbuild: 1.0.0

My coverage result shows the coverage generated from business and test projects. I have tried to add ExcludeFromCoverage attribute to all classes in test but no luck.

Here is ExcludeFromCoverage attribute class.

[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method | AttributeTargets.Constructor)]
public class ExcludeFromCoverageAttribute : Attribute
{
}

Any clue?

Thank you.

I'm brand new to coverlet and are having some issues with understanding how to exclude from coverage, the Readme.MD is not self explanatory and I have been unable to find any other doc.
In my project some are working with resharper to get coverage.

Hence, 2 things:

• Example on how to exclude e.g. a class from coverage with Coverlet
• Is it possible to reuse resharpers [ExcludeFromCodeCoverage]-markings?

currently I have set up VS codes testtask to run coverlet and are using Coverage Gutters to decorate the editor, seems to working just fine. However I would like a way to view coverage as an html-report, any ideas?

Thank you and hopefully these questions are not too awkward as an issue.

/E

First of all HUGE thank you for doing the heavy lifting for Cobertura directly in coverlet. This is a huge time saver and reduces the complexity of report conversion for Jenkins pipeline reporting. Now for the fun part :). As part of investigating usage of this format, I found the following components need to be updated/fixed in order to get the same (BETTER!) experience that we currently get with OpenCover -> Cobertura using the converter:

1. In the package node there is no name attribute, this should be the assembly name of the code being covered (without dll)
2. The filename section does not include the full path to the files being covered, which can break linkage in Jenkins (this could also be a jenkins specific problem, but wanted to call it out).
3. Coverage node is missing the following info:

• lines-valid
• lines-covered
• branches-covered
• branches-valid
• complexity

4. In line coverage, when it is a branch hit the line node does not include the attribute: condition-coverage
5. There is a discrepancy between how you are defining signature and how it gets converted, and I am not sure which is right: the converter reports a method like this:

• <method name=".ctor" signature="(OpenQA.Selenium.IWebDriver)" line-rate="1" branch-rate="1">
  vs. the built-in here in coverlet reports it like this:
  <method name=".ctor" signature="System.Void UI.Framework.Pages.BasePage::.ctor(OpenQA.Selenium.IWebDriver,System.Boolean)" line-rate="1" branch-rate="1">

That is all I have found, and again, huge thank you for all of this hard work! With these issues addressed we can finally switch off OpenCover -> Cobertura conversion!

Final note/information: I am using your latest coverlet package (1.1.0) and the latest OpenCoverToCoberturaConverter package (0.3.1). I then compared the cobertura xml from the converter to using cobertura directly in coverlet to get the info above.

https://github.com/tonerdo/coverlet/blob/c04fc782af3a77426e449f2b33c4c36fa738dd28/src/coverlet.msbuild/coverlet.msbuild.props#L7

^^ should have a condition as to not overwrite user provided values.

Lets say we have 3 projects.

Library
   |
   +--(Referenced by) Application
            |
            +--(Referenced by) Tests

In AppData\Temp, coverage is generated for "Application" but not for "Library", or even if it is generated, it fails with an error that could not find Library_SOME_GUID file. But there are multiple Application_SOME_GUID files in AppData\Temp.

I suspect, Library_SOME_GUID is written as Application_SOME_GUID

Hi,

I just upgraded to the latest v1.1.0 GA build. I'm using the new /p:Threshold=80 option to the build process yet the test run still returns 'successful' even if the coverage is below the threshold.

This is the command I run to build:

dotnet test --configuration Release --logger 'trx;logfilename=mstest-results.trx' /p:CollectCoverage=true /p:CoverletOutputFormat=opencover /p:CoverletOutputDirectory=TestResults /p:Threshold=80

This is the output:

Microsoft (R) Test Execution Command Line Tool Version 15.3.0-preview-20170628-02
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
[xUnit.net 00:00:01.5821864]   Discovering: UnitTests
[xUnit.net 00:00:01.7179983]   Discovered:  UnitTests
[xUnit.net 00:00:01.7245869]   Starting:    UnitTests
[xUnit.net 00:00:01.9276830]     AuthResponsesOperationFilter does not add 401 response to operation without AuthoriseAttribute [SKIP]
[xUnit.net 00:00:01.9282262]       Cannot yet mock ApiDescription objects
Skipped  AuthResponsesOperationFilter does not add 401 response to operation without AuthoriseAttribute
Standard Output Messages:
 Cannot yet mock ApiDescription objects
[xUnit.net 00:00:01.9862873]     AuthResponsesOperationFilter Adds 401 response to operation with AuthoriseAttribute [SKIP]
[xUnit.net 00:00:01.9863816]       Cannot yet mock ApiDescription objects
Skipped  AuthResponsesOperationFilter Adds 401 response to operation with AuthoriseAttribute
Standard Output Messages:
 Cannot yet mock ApiDescription objects
[xUnit.net 00:00:02.3872529]   Finished:    UnitTests

Total tests: 33. Passed: 31. Failed: 0. Skipped: 2.
Test Run Successful.
Test execution time: 3.8409 Seconds
WARNING: Overwriting results file: /home/jenkins/agent/workspace/an-updated-coverage-process-7N7FVRFUDPWSNYEY7B7OE6IM5VJ26KCY5NFVLTNHW4HURJ65KZAQ/tests/UnitTests/TestResults/mstest-results.trx
Results File: /home/jenkins/agent/workspace/an-updated-coverage-process-7N7FVRFUDPWSNYEY7B7OE6IM5VJ26KCY5NFVLTNHW4HURJ65KZAQ/tests/UnitTests/TestResults/mstest-results.trx

Calculating coverage result...
  Generating report 'TestResults/coverage.xml'

| Module                 | Coverage |
|------------------------|----------|
| ApplicationCore        | 75%      |
| Infrastructure         | 37.8%    |
| Nedbank.Authentication | 0%       |
| Web                    | 42.9%    |

Am I missing something or is this not working right now.

Also my build environment is:

Microsoft (R) Build Engine version 15.3.409.57025 for .NET Core
We're running .NET Core 2.0 on Linux. (Install .NET Core SDK 2.1.104 Package)

Hey,
I have seen the recent article on Hanselman about this extension. We are using .NET core extensively and we are stuck in test coverage. We are using Visual studio 2017 and using xUnit as test framework. Could you please provide us an extension for coverlet in Visual studio 2017 also. It would be good if you can provide for community, Professional and enterprises edition

First, great work! It's really great to get the Cobertura support.

Though, between 1.1.0 and 1.1.1 something happened which results in the following:

Calculating coverage result...
Generating report './coverage.xml'
...coverlet.msbuild.targets(19,5): error : String cannot be of zero length.
...coverlet.msbuild.targets(19,5): error : Parameter name: oldValue

I only get this for 1.1.1 with cobertura enabled. 1.1.0 works with cobertura enabled. 1.1.1 works fine with lcov.

I am guessing that basepath sent as source to GetRelativePathFromBase for some reason is empty?

Another thing. Maybe wrong forum/tool, but is there any way to limit the sources included in the coverage file? As of now, I get coverage result for 4 dll:s, I only need coverage report for one of them.

Thanks.

I get an error (warning?) printed when running coverlet - it looks like coverage results are still generated though:

/Users/user/.nuget/packages/coverlet.msbuild/1.0.1/build/netstandard2.0/coverlet.msbuild.targets(19,5): error : Attempted to divide by zero. [/Users/user/sln/tests/UnitTests/UnitTests.csproj]

Hi guys!

Why Coverlet use the custom attribute ExcludeFromCoverage from its own namespace instead of using already exists in .NET Core attribute ExcludeFromCodeCoverage from namespace System.Diagnostics.CodeAnalysis?

To Coverlet ignore any class, I need to install the nuget package coverlet.msbuild in all projects with tests, right?

I am getting the error

"System.IO.FileNotFoundException: No test is available in <myassembly.dll>. Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk" and framework version settings are appropriate and try again."

...when running a batch script from TFS that executes coverlet.

I have added a reference to the "Microsoft.NET.Test.Sdk" assembly to my project (it is added by default by Visual Studio).

My project is ASP.NET Core 2.0 Razor Pages
Visual Studio 2017 15.3
MSBUILD 15
Team Foundation Services 2015

It runs without any errors when executed directly from the build server from the command-line. The error only occurs when executed from our TFS build.

with the new 1.1.0 version i've a memory allocation issue. i tried it with one of my smaller projects. it generates almost a 10 GB hits-file. and it looks like it would need at least that much memory. currently the system runs out of memory and the dotnet process is stuck while trying to allocate more memory.

it looks like we don't get coverage reports for files which have same name and classes. For some APIs we we have multiple versions which only differ in a versioned namespace.. At the moment it looks like we get coverage report for the file ion namespace version1 but not in version2. Could there be some mapping issues when only the namespace is different?

It would be handy to have a way of preventing the file copies in situations where the build artefacts are throw away; i.e. Build systems.

File copying can take up a fair amount of time in a sufficiently large solution, and if you don't care about saving the original, there are 2 file copies that don't need to happen - the backup, and the restore. All in all, this could have a considerable time saving in build pipelines where the output is throw away anyway.

More of a question really, but I've noticed that a test run with a failed test in it does not produce any coverage summary. Is that expected? If so is there some way to change that behaviour?

I'm going to go and check if it does produce the coverage file. If it does produce that, it's not really an issue for me, but would be nice to have the behaviour synced.

Thanks.
Paul

Document that tests and tested assembly must be separate (no single assemblies)

Improvement. Add to average return of code coverage of all lines.
example:

project 1 | 100%
project 2 | 50%

Total coverage: 75%

Hi!

Thanks for the awesome tool You built up!

A have a quick question about ExcludeFromCoverage.
In the documentation, You mention specifying classes. Is it possible to exclude by namespaces? Is there an example I can reference?

so would the following example be valid to exclude all test files from coverage?

dotnet test MyProject.csproj /p:CollectCoverage=true /p:ExcludeFromCoverage=MyProject.Test.*

I've been using the report generator by danielpalme, to visualise the output from coverlet.

https://github.com/danielpalme/ReportGenerator

I get errors using this tool since upgrading from 1.0.2 to 1.1.0, I wonder if there is an error in the file being produced, I get the same error message if I try cobertura.

I have reverted back to 1.02 and the reporting tool works.

The error message from the reporting tool was:

Error in Report Generator: Unhandled Exception: System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> System.NullReferenceException: Object reference not set to an instance of an object. at Palmmedia.ReportGenerator.Parser.OpenCoverParser.GetBranches(XElement[] methods, HashSet1 fileIds)
at Palmmedia.ReportGenerator.Parser.OpenCoverParser.ProcessFile(HashSet1 fileIds, Class class, String filePath) at Palmmedia.ReportGenerator.Parser.OpenCoverParser.ProcessClass(Dictionary2 fileIdsByFilename, Assembly assembly, String className)
at Palmmedia.ReportGenerator.Parser.OpenCoverParser.<>c__DisplayClass12_0.b__9(String className)
at System.Threading.Tasks.Parallel.<>c__DisplayClass17_01.<ForWorker>b__1() at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask) at System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object ) --- End of inner exception stack trace --- at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action1 body, Action2 bodyWithState, Func4 bodyWithLocal, Func1 localInit, Action1 localFinally)
at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable1 source, ParallelOptions parallelOptions, Action1 body, Action2 bodyWithState, Action3 bodyWithStateAndIndex, Func4 bodyWithStateAndLocal, Func5 bodyWithEverything, Func1 localInit, Action1 localFinally)
at System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable1 source, Action1 body)
at Palmmedia.ReportGenerator.Parser.OpenCoverParser.ProcessAssembly(String assemblyName)
at Palmmedia.ReportGenerator.Parser.OpenCoverParser.<.ctor>b__5_5(String assemblyName)
at System.Threading.Tasks.Parallel.<>c__DisplayClass17_01.<ForWorker>b__1() at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask) at System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object ) --- End of inner exception stack trace --- at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action1 body, Action2 bodyWithState, Func4 bodyWithLocal, Func1 localInit, Action1 localFinally)
at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable1 source, ParallelOptions parallelOptions, Action1 body, Action2 bodyWithState, Action3 bodyWithStateAndIndex, Func4 bodyWithStateAndLocal, Func5 bodyWithEverything, Func1 localInit, Action1 localFinally)
at System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable1 source, Action1 body)
at Palmmedia.ReportGenerator.Parser.OpenCoverParser..ctor(XContainer report)
at Palmmedia.ReportGenerator.Parser.ParserFactory.GetParsersOfFile(String reportFile, ClassSearcherFactory classSearcherFactory, ClassSearcher globalClassSearcher)
at Palmmedia.ReportGenerator.Parser.ParserFactory.CreateParser(IEnumerable1 reportFiles, IEnumerable1 sourceDirectories)
at Palmmedia.ReportGenerator.Generator.GenerateReport(ReportConfiguration reportConfiguration)
at Palmmedia.ReportGenerator.Program.Main(String[] args)`

Hi.

First of all thanks a lot for Your initiative to write cross platform coverage library :) I've been using OpenCover and been dreaming about cross platform analogue :)

I have NUnit test project and try to integrate coverlet, but get an error:
...
Total tests: 13. Passed: 13. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 1.4340 Seconds

Calculating coverage result...
C:\Users\Alexander.nuget\packages\coverlet.msbuild\1.0.0\build\netstandard2.0\coverlet.msbuild.targets(19,5): error : Could not find file 'C:\Users\Alexander\AppData\Local\Temp\TicTacToe_1a57e94d-46ac-44ce-9320-99f8c74d6267'. [C:\Work\TicTacToe\test\TicTacToe.Test\TicTacToe.Test.csproj]

I tried Hanselman's project and it works well.
You can find my project here https://github.com/AlexanderButs/TicTacToe
Thoughts?
Thanks one more time for great job.

I assumed adding

  dotnet test  /p:CollectCoverage=true  /p:Threshold=100

would cause dotnet test to return a non zero result.

How do you use Threshold in project to fail the test run?

Hi. Firstly, thanks for your efforts. By far the best approach I've seen so far in this space.

I've got a rather large solution with many interdependencies between projects. When I run my tests, for almost all of the projects I see the following error:

tests/MyAssembly.Tests
/src/packages/coverlet.msbuild/1.0.1/build/netstandard2.0/coverlet.msbuild.targets(7,5): error : Failed to resolve assembly: 'AnotherAssembly, Version=0.1.0.0, Culture=neutral, PublicKeyToken=null' [/src/tests/MyAssembly.Tests/MyAssembly.Tests.csproj]

obviously sanitised.

AnotherAssembly isn't a direct dependency of the TestProject, but is built - to the version specified, and is in the bin directory along with all the other dependencies.

Any idea what might be wrong? Or better still, teach a bloke to fish: Is there some way of turning up the logging on coverlet to help debug? (can't actually say I've seen anything in the code which looks like it might help with this, but I haven't specifically been looking for it. will look now)

Cheers,
Paul

https://github.com/tonerdo/coverlet/blob/75a43d2a39a8733289875c3b00ebdbb64adc4385/src/coverlet.msbuild/coverlet.msbuild.props#L11

Output gets formatted with \ resulting in trying to write to a directory not supported by non-CMD terminals. Additionally, running without /p:CoverletOutputDirecotory results in Project file does not exist error.

Not sure how to work around this, or if it's possible given that this value is hardcoded.

Is it possible to use this in a situation that unit tests does not exist?

I have an asp.net core app and I want to measure its coverage based on its actual usage in my prod environment.

Just noticed this in my build pipeline

...
NUnit Adapter 3.8.0.0: Test execution complete

Total tests: 163. Passed: 144. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 2.4027 Minutes
Results File: /tmp/build/5128ae76/.output/_85bded69-6543-4650-7e86-6797d5d60ecd_2018-04-11_12_48_55.trx

Calculating coverage result...

real	5m55.490s
user	3m20.960s
sys	0m22.580s

Basically, ran 144 tests, but obviously failed silently when generating the coverage file. This happened on a number of test assemblies. :-(

Any ideas what might be causing that. I've not been able to see that happen locally.

If I create a coverage file in the LCOV format - how do I consume in Sonarqube. I tried the following in my sonar-project.properties file sonar.javascript.lcov.reportPath=/coverage.info - but this seems wrong as this is a .net core cs project. It is not being picked up by sonarqube.

Thanks for implementing the much-needed functionality.
Do you have any plans/timelines for implementing it as a console runner? so that we won't require adding nuget in all project and can be used as a tool to generate the coverage report.

When excluding a whole project (using /Exclude flag, but I'm guessing after reviewing the code this could happen if using the label exclusion method as well for all project)

1. the project is still showing in the report with NaN% percentage
2. The overall percentage of the report isn't calculated correctly and/or the threshold calculations aren't not correct.

To reproduce:
dotnet test test\coverlet.core.tests\coverlet.core.tests.csproj -c Release /p:CollectCoverage=true /p:CoverletOutputFormat=opencover /p:Exclude=\"../../src/coverlet.core/\"

If you had this to the build.proj before creating the nugt package it would be easier to reproduce:
<Target Name="RunTestsWithExclude" AfterTargets="RunTests"> <Exec Command="dotnet test --no-build $(MSBuildThisFileDirectory)test\coverlet.core.tests\coverlet.core.tests.csproj -c $(Configuration) /p:CollectCoverage=true /p:CoverletOutputFormat=opencover /p:Exclude=\&quot;../../src/coverlet.core/,$(MSBuildThisFileDirectory)/src/coverlet.core/Reporters/**/*.cs\&quot;"/> </Target>

The output:

Expected output:

or

Also the project should not be accounted for as part of the overall threshold calculations (in case there are other projects which are included)

When I run command
dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=opencover
all test are passing and code coverage is correctly generated which is good. However returned exit code from command is 1 due to this error:

coverlet.msbuild\1.0.2\build\netstandard2.0\coverlet.msbuild.targets(13,5): error : Failed to resolve assembly: 'Common, Version=1.1.0.0, Culture=neutral, PublicKeyToken=null' [C:\Source\Repos\MyApp\src\Test\Test.csproj]

Where Common is one of the referenced projects.
When I just run dotnet test everything is working and exit code is 0.

My coverage script does the following:

...
# Run the tests
    for testproj in $(ls tests/*.Tests -d | grep -v Integration); do 
        echo ${testproj}; 
        set +e
        time \
            dotnet test \
                --no-build \
                --no-restore \
                --filter "TestCategory!=Integration" \
                --logger:trx \
                -r ${base_dir}/.output/ \
                $testproj \
                /nowarn:NU1701,NU1603,NU1605 \
                /p:CollectCoverage=true \
                /p:CoverletOutputFormat=opencover
        cname=$(basename $testproj)
        cp ${testproj}/coverage.xml ${base_dir}/.output/$cname-coverage.xml
        set -e
    done
...

This works perfectly well for the most part, but I see the following error randomly in different parts of the output on each run:

...
Total tests: 18. Passed: 18. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 4.4549 Seconds
Results File: /tmp/build/5128ae76/.output/_161d9565-a576-4c2a-7fe5-1bb0ae8a6598_2018-04-04_12_54_05.trx

Calculating coverage result...
/tmp/build/5128ae76/source-code-core/packages/coverlet.msbuild/1.0.3-beta1/build/netstandard2.0/coverlet.msbuild.targets(19,5): error : The process cannot access the file '/tmp/build/5128ae76/source-code-core/tests/Diagnostics.Tests/bin/Debug/netcoreapp2.0/MyAssemblyDependency.dll' because it is being used by another process. [/tmp/build/5128ae76/source-code-core/tests/Diagnostics.Tests/Diagnostics.Tests.csproj]

real	0m6.404s
user	0m2.980s
sys	0m0.280s
cp: cannot stat 'tests/Diagnostics.Tests/coverage.xml': No such file or directory

Any ideas?

Cheers,
Paul

NB: coverlet-1.0.3-beta1 is a build of the current master so as to capture the change from PR #23 and hosted in a local nuget repo.

Hi, we would love to start using coverlet to generate code coverage, but we really don't want to include code that gets automatically generated in our coverage results. Since we generate it during CI, it wouldn't be feasible for us to apply attributes to every class that we want to exclude from coverage. We do, however, have a common folder in which the code gets generated.

Is there a flag we can pass to exclude the files in said folder from coverage?

I'm trying to use coverlet with the NUnit3TestAdapter package. However, I get an error message telling me that coverlet.core does not have a strong name. Is it possible to sign the package?

I created the following test project:

<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
  <TargetFramework>net461</TargetFramework>
</PropertyGroup>

<ItemGroup>
  <PackageReference Include="coverlet.msbuild" Version="1.0.2" />
  <PackageReference Include="NUnit" Version="3.10.1" />
  <PackageReference Include="NUnit3TestAdapter" Version="3.10.0" />
</ItemGroup>

</Project>

The project contains one file, Tests.cs:

using NUnit.Framework;

namespace SimpleTestProject
{
    
    public class Tests
    {
        [Test]
        public void FailingTest()
        {
            Assert.That(false);
        }

        [Test]
        public void SucceedingTest()
        {
            Assert.That(true);
        }
    }
}

If I run the tests using dotnet test I get the following result:

Build started, please wait...
Build completed.

Test run for C:\Users\Maarten\Source\Repos\Allego.Testing.NUnit.Coverage\SimpleTestProject\bin\Debug\net461\SimpleTestProject.dll(.NETFramework,Version=v4.6.1)
Microsoft (R) Test Execution Command Line Tool Version 15.6.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
NUnit Adapter 3.10.0.21: Test execution started
Running all tests in C:\Users\Maarten\Source\Repos\Allego.Testing.NUnit.Coverage\SimpleTestProject\bin\Debug\net461\SimpleTestProject.dll
NUnit3TestExecutor converted 2 of 2 NUnit test cases
NUnit Adapter 3.10.0.21: Test execution complete
Failed   FailingTest
Error Message:
   Expected: True
  But was:  False

Stack Trace:
   at SimpleTestProject.Tests.FailingTest()


Total tests: 2. Passed: 1. Failed: 1. Skipped: 0.
Test Run Failed.
Test execution time: 1,4818 Seconds

However, when I use /p:CollectCoverage=true, I get the following output:

Build started, please wait...
Build completed.

Test run for C:\Users\Maarten\Source\Repos\Allego.Testing.NUnit.Coverage\SimpleTestProject\bin\Debug\net461\SimpleTestProject.dll(.NETFramework,Version=v4.6.1)
Microsoft (R) Test Execution Command Line Tool Version 15.6.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
An exception occurred while invoking executor 'executor://nunit3testexecutor/': Exception has been thrown by the target of an invocation.
Could not load file or assembly 'coverlet.core, Version=1.0.2.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. A strongly-named assembly is required. (Exception from HRESULT: 0x80131044)

Kind regards,

Maarten van Sambeek

it works quite well so far! we are using it already in VSTS/TFS. but one thing which is not so streamlined for using in VSTS/TFS are the supported output formats. Currently VSTS/TFS only supports Cobertura and JaCoCo. Supporting one of this formats would be great.

The coverage report for 87135a5 claims that DictionaryExtensions.cs has 75% coverage. It only has one method:

https://github.com/tonerdo/coverlet/blob/87135a5d0c3c5a27db83509fba01774dd20201d9/src/coverlet.core/Extensions/DictionaryExtensions.cs#L7-L14

According to the report, lines 9, 10 and 13 are covered but not line 12 (thus the 75%). How can 13 be covered but not 12? Is this a bug or am I misreading something? The only test for this method…

https://github.com/tonerdo/coverlet/blob/87135a5d0c3c5a27db83509fba01774dd20201d9/test/coverlet.core.tests/Extensions/DictionaryExtensionsTests.cs#L10-L17

…seems to cover it 100%.

I have 3 projects and 21 tests all written against Candle.Services

Candle.DbContext
   |
   +--(References) Candle.Services
   |    |
   +----+--(References) Candle.Services.Tests

Calculating coverage result...
  Generating report 'C:\..\Candle.Services.Tests\coverage.json'

+------------------+----------+
| Module           | Coverage |
+------------------+----------+
| Candle.DbContext | 21%      |
+------------------+----------+
| Candle.Services  | 0%       |
+------------------+----------+

Hello!

I have multiple tests projects - one with unit tests and one with integration tests. How can I calculate code coverage for both projects with coverlet?

Use --option in addition to or instead of property notation /p:

/p:CollectCoverage=true -> --collect|-c (true|false)
/p:CoverletOutputFormat=opencover -> --output|-o (json (default), lcov and opencover)
/p: CoverletOutputDirectory --> --directory|-d

Add gif from MVP Conference presentation to readme.

Maybe related: #39

Behavior.

What I do

1. Add coverlet.msbuild NuGet to my test project.
2. Run Tor (needed for my tests to successfully complete.)
3. Run dotnet test /p:CollectCoverage=true.

I also tried creating a new project and everything works fine, however with my current project (https://github.com/zkSNACKs/WalletWasabi/) it silently fails.

The back story to this is that I use VSTS for builds. There is out of the box functionality for reporting code coverage totals in the build summary when using the cobertura format. However I'd also like to use the tool fdesjardins/coverage-viewer to generate a nice html report and it requires the standard json output.

I'm happy to do some work in this space but before I start I wanted to see what your opinion was on this.

I figure a nice way to access this functionality would be to allow comma separated values for CoverletOutputFormat. So in my case that would look like /p:CoverletOutputFormat=json,cobertura.

Looking at the code it should be pretty straight forward to implement.

Expected Result

When running code coverage on a project, all branches in a method will be determined, and their targets will be instrumented.

Actual Result

Currently, only sequence points that are targets of branches are instrumented.

Proposed Solution

Implement a branch point/target search using logic similar to OpenCover.Framework.Symbols.CecilSymbolManager. This would allow us to:

• Track Branches separately from Sequence Points.
• Instrument all Branch Targets.
• Provide more detailed branch information to the supported reporters.

Discussion

I've already taken an initial stab at this functionality in my fork and everything seems to be working. I am in the process of cleaning up the code and adding tests, but wanted to open this issue before opening a PR.

Major high-level changes to the structure are as follows:

• Removed IsBranchTarget from Line and LineInfo as branches are now tracked separately from sequence points.
• Changed InstrumenterResult
  • Added Branch which extends Line and adds additional properties such as Path
  • Changed Document to have both Lines and Branches
• Changed CoverageResult
  • Updated Methods to be a Dictionary with a new class Method as the Value
  • Method contains Lines and Branches
• Changed all supporting code (Instrumenter, CoverageSummary, Reporters) to handle the new model structure.

If you check out a generated report for lcov you will see stuff like this:

Format should probably look more like this https://github.com/alexandrudima/vscode-lcov/blob/master/examples/fizzbuzz/coverage/lcov.info

First, not to be too patronizing, but I've loved the experience I've had with Coverlet so far while unit testing. I've introduced it into a number of projects I've worked on, and it's helping me to identify pathways that really ought to have unit tests.

That being said, there are sometimes pathways that I can't unit test. I'd like to be able to measure whether or not a suite of acceptance tests covers those pathways. After poking around a bit, I figured it would be feasible:

• In the executable hosting the acceptance tests (which references a second project), add a reference to coverlet.msbuild and additional build targets to run the compiled .dll, and then generate the coverage result from that run.

  <Target Name="CodeCov" AfterTargets="InstrumentModulesAfterBuild">
    <Exec Command="dotnet quot;$(MSBuildThisFileDirectory)$(OutputPath)AcceptanceTests.dll&quot; --diagnostics" />
  </Target>
  <Target Name="GenerateCoverageResult" AfterTargets="CodeCov">
    <Coverlet.MSbuild.Tasks.CoverageResultTask Condition="$(CollectCoverage) == 'true'" Output="$(CoverletOutput)" OutputFormat="$(CoverletOutputFormat)" />
  </Target>

• Run dotnet msbuild .\AcceptanceTests.csproj /t:CodeCov /p:CollectCoverage=true /p:CoverletOutputFormat=opencover

That much generates the instrumentation (verified in dotPeek) and begin the application. After the first line of code is executed, however, I get this error:

  Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'coverlet.core, Version=1.1.1.0, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.

I've tried adding references to the projects, adding additional probing paths, and setting binding redirects, to no avail. Is there a way to support this functionality, or perhaps a better question, is this something that Coverlet should allow at all?

I am trying to add code coverage to kubernetes-client-csharp.

Desperate of not having official support from Microsoft I am giving a try to coverlet. I've created a feature branch from project's master and added Coverlet (everything fine in Travis so far.

But, as soon as I enable the "/p:CollectCoverage=true", I start receiving the following error in 80% of tests we execute:

[xUnit.net 00:00:01.7251872]       System.BadImageFormatException : [/home/travis/build/sesispla/kubernetes-client-csharp/tests/bin/Debug/netcoreapp2.0/KubernetesClient.dll] No string associated with token.

Full log from faulty TravisCI job

We're currently invoking the code coverage with this bash script:

#!/usr/bin/env bash

# Exit on any error
set -e

# Ensure no compile errors in all projects
dotnet restore
dotnet build --no-restore

# Execute Unit tests
cd tests
dotnet test --no-restore --no-build /p:CollectCoverage=true /p:CoverletOutputFormat=lcov /p:CoverletOutput=./lcov.info
if [[ $? != 0 ]]; then
    exit 1
fi

Also happens locally with MacOS 10.12.6 High Sierra.

If I set /p:CollectCoverage=false tests passes properly in both TravisCI and locally.

Any clue on what's going on?

Thanks you!

This is a minor issue, but it causes other programs that read the xml file to fail with the following error:
There is no Unicode byte order mark. Cannot switch to Unicode, since it expects the file to actually be UTF-16, even though it is saved as UTF-8. My recommendation is to change the header declaration to either remove the encoding piece, or specify UTF-8

One of our build runners is Ubuntu-based currently, building a reasonably large project (~15 projects in a single solution). Using dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=opencover, I'm seeing some files created in /tmp that are fairly enormous (naming scheme <project>_<hash>) - one of the files covering our largest project is nearly 90gb - about 180gb for all projects combined. Disabling coverlet, the files don't get created at all, so I'm assuming they're cache files of some kind.

Should they be this big? Our build runner doesn't have that much disk space allocated, so it keeps bringing it down :|

Hey,
So I'm having an issue where it appears as though only the first DLL in the output build is being instrumented. We have 5 dll's being output in a linux .net core project. Coverage works for 1 dll (alphabetical first from what I can tell) but the other dll's are all being reported as 0 coverage.

Cheers,
Jason

• Filter modules to be instrumented

@tonerdo IMHO could be great avoid attribute instrumentation for filter out/in class to test. You/We could think a command switch/json file format to pass.
However...great work compliments.