Comments (10)
Alright, I debugged the changes and finishing up all the additions for these requirements of code coverage. Here is a link to the PR, I included some tests and docs to make sure it works as expected. I'll describe the scenarios that I tried to include (I think I have all the ones you outlined form this issue). If you see any issues or have any suggestions, let me know @jhornsb2 :)
from unity-test-runner.
@nicholas-maltbie I just got the chance to look through all of the changes. Thank you for implementing it. It looks great and should cover exactly what I need.
from unity-test-runner.
I've been looking at the repo and it looks like it could probably be added as a flag to the end of this file, https://github.com/game-ci/unity-test-runner/blob/main/dist/steps/run_tests.sh
Maybe add a new flag like "-enableCodeCoverage"
And at if both "$PLAY_MODE" = "true" and "$EDIT_MODE" = "true" and the code coverage flag is enabled (as a separate clause added at the end), we could add a clause to add custom flags based on if code coverage is included.
I might fork this repo tomorrow to see if I can get it working on my own project for validation. I'll post an update if I come up with anything.
from unity-test-runner.
I'm working on drafting a PR for this addition right now, my first time using node in a while so code might be a bit off.
I don't think it will work yet (still need to test it) but working on just setting out the boiler plate code with three parameters
- enableCodeCoverage (optional flag, default of disabled)
- assemblyFilters (optional, requires enableCodeCoverage)
- coverageResultsPath (optional, requires enableCodeCoverage)
https://github.com/nicholas-maltbie/unity-test-runner/tree/nicholas-maltbie/codecoverage-support
I'll work on testing it, but have to get on a flight soon so will probably have some update in the next few days! hopefully can test it with my project soon.
from unity-test-runner.
Oh, I was planning on working on the solution for this myself. I was just checking to see if this was a desired change since there have been other issues and pull requests related to code coverage.
from unity-test-runner.
@nicholas-maltbie, @davidmfinol Thanks for your work on the code coverage. I tried to run game-ci/unity-test-runner@v2 in a self-hosted windows runner. I did not provide coverageOptions parameter so the default value is used. I can see the tests are run successfully and the test artifact result is expected. Also, code coverage folder is generated with content. However, I got 0% coverage for all files. When I run the test from the cmd with the same set of parameters locally I get the expected code coverage result which is not 0% for some files. Do you guys have any idea why this is happening? Thanks.
The image below is the command list captured from game-ci/unity-test-runner@v2 log which is the same as the one I used locally except for the path difference.
from unity-test-runner.
Hi @spencer-cheng , i ran into this issue as well. I found that explicitly including the assemblies I want to generate coverage options for such as 'assemblyFilters:+my.assembly.*' may help. Are there coverage files generated in the output folder that or is the result just empty?
from unity-test-runner.
Hi, @nicholas-maltbie Thanks for your reply. I tried to explicit include them. But it does not help.
I did more tests in the past two days. It works when i run it on "ubuntu-latest".
However the same code does not when i choose "windows-2019". Similarly, if i run it using a self-hosted windows 2019 runner, the action does not work. If i change to use "run unity command line directly" in step instead of using unity-test-runner action, the coverage data is generated correctly as expected.
I guess there might be a problem with code coverage's windows implementation. Thanks.
from unity-test-runner.
@spencer-cheng , there are a few differences between unity when it runs on windows versus Linux in CLI mode. Are you able to produce coverage files properly when running on your local windows machine via cli directly? It might be good to isolate if it's a windows problem when running with CLI or a problem with the way the test runner is configured.
If you're able, could you share the configuration you're using to generate test coverage? Having this workaround of using the run unity command line directly
may prove useful to others if this issue takes a while to resolve for windows containers.
from unity-test-runner.
@nicholas-maltbie I am able to produce coverage files properly when running on my local windows machine via CLI directly.
My workaround is like:
- install Unity on the Windows self-hosted runner and activate the license
- instead of using unity-test-runner action, i use a step like
- name: run tests directly from unity
shell: pwsh
run: |
cd dir/path/to/run_test.bat
./run_test.bat ${{ env.unity_exe_path}} ${{env.TARGET_TEST_DIR}} ${{env.TARGET_COVERAGE_DIR}}
echo "::set-output name=artifactsPath::${{ env.TARGET_TEST_DIR}}"
echo "::set-output name=coveragePath::${{ env.TARGET_COVERAGE_DIR}}"
run_test.bat is like (depends on where the bat file is, the relative path needs to be updated):
"%1" -batchmode -nographics -logFile "..\..\..\%2\playmode.log" -projectPath "..\..\..\3DMonitoringSystem" -runTests -testPlatform playmode -testResults "..\%2\playmode-results.xml" -debugCodeOptimization -enableCodeCoverage -coverageResultsPath "..\%3" -coverageHistoryPath "..\%2" -coverageOptions "generateAdditionalMetrics;generateHtmlReport;generateHtmlReportHistory;generateBadgeReport;assemblyFilters:-Assembly-CSharp-Editor,-My.Assembly;pathFilters:+*/Assets/_Project/*,+*/Assets/InputSettings/*"
from unity-test-runner.
Related Issues (20)
- Package Testing Hangs if Package Being Tested is the Root Directory of the Repository HOT 2
- Feature: Support Windows Runners for Package Testing
- Feature: Support Package Testing for Packages with Dependencies Outside of the Unity Registry HOT 1
- Feature: Add Caching for Package Testing
- Feature: Allow `unityVersion` of `auto` for Package Testing
- Code Coverage does not write any output anywhere HOT 2
- Unity no longer supports manual activation of Personal licenses HOT 2
- Create breaking change that updates rolling version number from 1 to 2
- Inconsistent licence activation HOT 7
- Reasource not Available by Integration HOT 5
- 'Unclassified error occured while trying to activate license.' HOT 9
- cannot access '/github/workspace/CodeCoverage': No such file or directory HOT 5
- v4.1.0 causing activation to fail when using manual offline activation HOT 5
- fullProjectPath is not set when using runAsHostUser
- packageMode workflow doesn't run playmode tests HOT 2
- chmod: cannot access '/github/workspace/CodeCoverage' HOT 1
- Licensing fails inconsistently during test step HOT 1
- Feature: Add support for Nuget dependency resolution when testing packages HOT 1
- Fails when organization or repository name have uppercase letters HOT 1
- Unity 6 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from unity-test-runner.