Comments (12)
Hi!
I have exactly the same issue with an extra twist. After switching .NET 8 the filenames changed from c:\users\me\...\ProjectA\ClassA.cs
to https://github.com/.../ProjectA/ClassA.cs
. The additional problem on my side is that these filenames are both present in the same coverage.cobertura.xml
.Most of them is the new https
version but some of them still refer to my filesystem. Unfortunately we use some futher tools to process these files (for example reportgenerator) and the calculated coverage is incorrect due to a inconsistent filenames.
I do some research and I found the following:
-
New filenames: it is the result of a breaking change in .NET 8 more information can be found here and here. So in this case I think when
UseSourceLink
option is turned on this is a normal behaviour. However in this method https://github.com/coverlet-coverage/coverlet/blob/master/src/coverlet.core/Reporters/CoberturaReporter.cs#L216 when source links are disabled it returns only the filename (probably it is correct) but comparing to previous .NET versions I have the feeling that source links are mixed up with full physical paths. I don't know but maybe a clarification would be great. -
About the mixed filenames: After a few test and detailed logging I think I found the problem: https://github.com/coverlet-coverage/coverlet/blob/master/src/coverlet.core/Coverage.cs#L368 between line 368 and 387. If there is no hits for an assembly the sourcelink is not used but I think this is not a correct or expected behaviour. If we reference a not covered assembly but would like to use sourcelinks the filename attribute should refer to them as well in the output file. A naive solution could be to change the order of the two
if
condition but I'm not familier with this project so I don't know what could be the consequences.
from coverlet.
Please use latest coverage.collector V6.0.2 package and replicate the issue.
from coverlet.
@Bertk I used the latest version.
from coverlet.
@Bertk
I set up a small project which demonstrates the issue: https://github.com/tungi52/CoverletBug
In the TestResults folder you can see the problem. You can also reproduce the issue just clone the repo then change the SDK version in the global.json
file (not forget to install both .NET6 and .NET8 SDK on your system) and run either
dotnet test CoverletBug_net6.sln --collect:"XPlat Code Coverage" --settings TestProject/test.runsettings
or
dotnet test CoverletBug_net8.sln --collect:"XPlat Code Coverage" --settings TestProject/test.runsettings
Hope this helps.
from coverlet.
Thank you for the repo but I was not able to reproduce the issue on my system. I noticed some generated code in your TestResults/with_sdk6/coverage.cobertura.xml
file which is not available on my system.
I used the following sequence to create the output in artifacts.zip.
dotnet restore C:\GitHub\coverlet-issue-1658\CoverletBug_net6.sln
dotnet build C:\GitHub\coverlet-issue-1658\CoverletBug_net6.sln
dotnet test TestProject\TestProject_net6.csproj -c Debug --collect:"XPlat Code Coverage" -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=cobertura
reportgenerator -reports:"TestProject/**/*.cobertura.xml" -targetdir:"artifacts\reports.cobertura.net6.0" -reporttypes:"HtmlInline_AzurePipelines_Dark;Cobertura"
dotnet clean
rd /S /Q TestProject\TestResults
=> "adapt global.json for net8.0"
dotnet restore C:\GitHub\coverlet-issue-1658\CoverletBug_net8.sln
dotnet build C:\GitHub\coverlet-issue-1658\CoverletBug_net8.sln
dotnet test TestProject\TestProject_net8.csproj -c Debug --collect:"XPlat Code Coverage" -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=cobertura
reportgenerator -reports:"TestProject/**/*.cobertura.xml" -targetdir:"artifacts\reports.cobertura.net8.0" -reporttypes:"HtmlInline_AzurePipelines_Dark;Cobertura"
from coverlet.
@Bertk did you clone my repo or just download it? Because the issue is reproducable only in the first case.
from coverlet.
I cloned the repository today and used the configuration combinations Format&SingleHit, Format&UseSourceLink, Format&IncludeTestAssembly, Format&SkipAutoProps.
I also used the provided test.runsettings file.
dotnet test TestProject\TestProject_net6.csproj -c Debug --collect:"XPlat Code Coverage" --settings TestProject\test.runsettings
I still cannot reproduce the issue on my system.
from coverlet.
the issue is only for .net 8. Did you try running this too on the cloned repo?
dotnet test TestProject\TestProject_net8.csproj -c Debug --collect:"XPlat Code Coverage" --settings TestProject\test.runsettings
from coverlet.
Finally, this issue is not related to cobertura coverage file format but the UseSourceLink option and coverlet.collector uses the local file name for net6.0 because UseSourceLink is not activated for net6.0 project files.
There is a .NET 8.0 breaking change Source Link included in the .NET SDK
The filename without activated UseSourceLink option.
<package name="CoveredClassLibrary_net8" line-rate="1" branch-rate="1" complexity="1">
<classes>
<class name="CoveredClassLibrary.CoveredClass" filename="CoveredClassLibrary\CoveredClass.cs" line-rate="1" branch-rate="1" complexity="1">
<methods>
<method name="Sum" signature="(System.Int32,System.Int32)" line-rate="1" branch-rate="1" complexity="1">
<lines>
<line number="5" hits="1" branch="False" />
</lines>
</method>
</methods>
<lines>
<line number="5" hits="1" branch="False" />
</lines>
</class>
</classes>
The filename with activated UseSourceLink option.
<package name="CoveredClassLibrary_net8" line-rate="1" branch-rate="1" complexity="1">
<classes>
<class name="CoveredClassLibrary.CoveredClass" filename="https://raw.githubusercontent.com/tungi52/CoverletBug/89244e69a571cda8ff30e53aa8aa4760e0985249/CoveredClassLibrary/CoveredClass.cs" line-rate="1" branch-rate="1" complexity="1">
<methods>
<method name="Sum" signature="(System.Int32,System.Int32)" line-rate="1" branch-rate="1" complexity="1">
<lines>
<line number="5" hits="1" branch="False" />
</lines>
</method>
</methods>
<lines>
<line number="5" hits="1" branch="False" />
</lines>
</class>
</classes>
Please
from coverlet.
Please aware that if a library is fully uncovered with UseSourceLink option the tool generates different filenames in the same report:
https://github.com/tungi52/CoverletBug/blob/main/TestResults/with_sdk8/coverage.cobertura.xml#L9
https://github.com/tungi52/CoverletBug/blob/main/TestResults/with_sdk8/coverage.cobertura.xml#L25
from coverlet.
@tungi52 Please create a separate issue for the inconsistent file name of uncovered library with enabled SourceLink configuration.
About the mixed filenames: After a few test and detailed logging I think I found the problem: https://github.com/coverlet-coverage/coverlet/blob/master/src/coverlet.core/Coverage.cs#L368 between line 368 and 387. If there is no hits for an assembly the sourcelink is not used but I think this is not a correct or expected behaviour. If we reference a not covered assembly but would like to use sourcelinks the filename attribute should refer to them as well in the output file. A naive solution could be to change the order of the two if condition but I'm not familier with this project so I don't know what could be the consequences.
The proposed update could have an performance impact and need further investigations.
from coverlet.
Newer issue #1679 addresses inconsistent filenames in coverage files with activated SourceLink option.
from coverlet.
Related Issues (20)
- [BUG] Upgrading v6.0.1 to v6.0.2 increases instrumentation time HOT 7
- Expected behavior when using CreateHardLinksForCopyLocalIfPossible HOT 1
- [BUG] Throw exception via method causes missing coverage HOT 2
- Question: Active logging or suggestions how to investigate a strange functionality HOT 1
- Records without empty primary constructor are not included in code coverage
- Coverage can be merged with `dotnet-coverage` tool
- Coverlet file not being created HOT 1
- 0% code coverage and error in collector HOT 1
- [BUG] coverlet generates coverage report for 'Microsoft.ApplicationInsights' package after upgrading to MSTest.TestAdapter 3.2.0 HOT 1
- New Version release
- No or incorrect coverage on expression body members
- Allow to set root directory for ExcludeAssembliesWithoutSources heuristic or other way to exclude internal nuget dependencies on shared CI nodes. HOT 3
- Usage of UseSourceLink results in 500 errors in Azure DevOps. HOT 1
- coverlet nuget package metadata for assemblies is missing/invalid HOT 1
- Coverlet doesnt calculate code coverage for tests until another test is added
- Holding a thread longer open then a test runs through causes coverlet to problably crash internaly and dont calc code coverage HOT 1
- [BUG] Inconsistent filenames with UseSourceLink after .NET 8
- [BUG] System.IndexOutOfRangeException using coverlet collector HOT 5
- Unable to exclude files using .runsettings in dotnet test. HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from coverlet.