Comments (16)
Finally finished a proof of concept here (you can download attached artifact and see yourself):
https://github.com/Macoron/unity-test-runner/pull/3/checks?check_run_id=447950614
So it's kinda work. Though there are several problems:
- HTML output missing several keys (looks like it's because some internal Unity CI setup). Also error messages looks weird with [CDATA] tags
- Need some way to distribute binary files (compiled reporter DLL and html template). Maybe nuget will work? Or use docker? Not sure, to be honest.
- The original repo has no code license. According to github rules it adds a lot of headache (read-only, no changes, etc). Already created issue to request licence.
In ideal situation it's need to be implemented as independent dockered Github Action (not as part of unity-testrunner). I have no experience in that, so will just include it as is in my CI pipeline.
Yeah, not perfect, but a good start.
from unity-test-runner.
Hey, thanks for a quick merge!
Unity Reporter distributes as compiled dotnet .dll which can be run on ubuntu by this action. I think, the most easiest way to setup CI pipeline is:
- Finish tests
- Install dotnet SDK 2.0 (or even better dotnet runtime 2.0)
- Run reporter
- Send html report as an artifact
I don't know should it be part of unity tester or separate repo. I'm kinda new in CI, so not sure which option is better.
Right now I'm testing how robust Unity Reporter it actually is.
from unity-test-runner.
Nice, I found much better solution called Extent Reports. It has very nice UI and Apache 2.0 license.
Made a few tweaks in my fork to make it run on .Net Core. And now it can generate awesome .html reports:
You can check modified workflow here. Download artifacts to see generated html. As I mentioned before, if someone will manage wrap it in dockered action - you will be a real hero.
from unity-test-runner.
Right now we use Extent Reports (described above). It works great and was easy to setup. I compiled it for .NET Core and placed binary files inside repository. Here is example of the test run and here is workflow.
Like I said before, more clever solution will be wrap it all in docker action with extent binaries and .NET inside. But this works too.
from unity-test-runner.
I no longer think we should implement Extent or use it as the recommended option (even though it looks nice at first sight).
Instead we should incorporate the official coverage tool (which includes generating reports) from Unity. See the changes in linked example above.
from unity-test-runner.
I don't think you're missing much. It's always a bit hard to find the right documentation for Unity related topics.
That said, it's at version 0.3 last time I checked, so it's also understandable.
You would have to pass a few flags to the commandline, similar to what @Fehrox tried to do here and similar to what Arthur Aulicino implemented here.
To me coverage reports are a lot more important than an overview which tests failed to be honest.
Hope that helps.
from unity-test-runner.
Having a look at it again; it doesn't seem like a very popular tool. Last commits were in March 2019, the readme is bloated and it only has a few stars.
https://github.com/Unity-Technologies/UnityTestRunnerResultsReporter
That said, there doesn't have to be anything wrong with it. It is probably a pretty straight forward tool. Curious to hear your findings!
from unity-test-runner.
Yeah, I found it only because Unity Dev point it to me (see this forum thread). I tested it a bit, It kinda work, but not perfect (see the attached report).
There are some errors, missing vars, etc. And I'm not sure why they need .log file, can't see where it used in actual report. But it's a good start.
from unity-test-runner.
My two goto tools for coverage reporting are Coveralls and CodeCov. There are also C# solutions for Coveralls and CodeCov also with an (example).
Unfortunately both seem to use OpenCover which seems to be Windows only. Also doesn't seem to be very active anymore.
I also found coverlet which seems to be promising and offers cross-platform compatibility. In it's repository there is a thread where Unity is being mentioned too.
In that thread, there's also a visual tool being suggested, called ReportGenerator which itself hints at multiple test frameworks that offer code coverage output.
For me this is a bit out of scope, but i'm willing to accept merge requests if anyone finds a good solution that works for everyone!
from unity-test-runner.
That looks nicer indeed!
What stops you from installing it as a nuget package in the docker image?
What are the things you have considered?
from unity-test-runner.
@webbertakken I just have zero experience in dockers. Maybe it's not that hard, but don't have dedication right now to setup properly.
from unity-test-runner.
This also looks to be interesting / promising: https://docs.unity3d.com/Packages/[email protected]/manual/UsingCodeCoverage.html#enable-code-coverage
from unity-test-runner.
@Macoron What did you end up using as your solution?
from unity-test-runner.
Unity has updated their codecoverage
package to 0.3.
See also an example implementation of it at gableroux/unity3d-gitlab-ci-example#42.
from unity-test-runner.
Seems like extent
should be a .NET Core Tool, not a regular nuget package. Regular nuget packages are meant to be written into a .csproj file and referenced in code as far as I'm aware.
If extent
was a .NET Core Tool, workflows could use the actions/setup-dotnet
like so:
- uses: actions/setup-dotnet@v1
with:
dotnet-version: '3.1.100'
- run: |
dotnet tool install -g extent
extent -i <test-results>.xml -o <test-results>.html
(edit) I suggested some changes to this end: Update project to .NET Core
from unity-test-runner.
@webbertakken Hi, I'm confused as to how Unity's code coverage tool can generate test reports. I can't find anything in the documentation that really describes the same functionality that extent does; it appears the code coverage reports soley report on code coverage with no test results.
Am I missing something?
from unity-test-runner.
Related Issues (20)
- Code Coverage does not write any output anywhere HOT 2
- Unity no longer supports manual activation of Personal licenses HOT 2
- Create breaking change that updates rolling version number from 1 to 2
- Inconsistent licence activation HOT 7
- Reasource not Available by Integration HOT 5
- 'Unclassified error occured while trying to activate license.' HOT 9
- cannot access '/github/workspace/CodeCoverage': No such file or directory HOT 5
- v4.1.0 causing activation to fail when using manual offline activation HOT 5
- fullProjectPath is not set when using runAsHostUser
- packageMode workflow doesn't run playmode tests HOT 2
- chmod: cannot access '/github/workspace/CodeCoverage' HOT 1
- Licensing fails inconsistently during test step HOT 1
- Feature: Add support for Nuget dependency resolution when testing packages HOT 1
- Fails when organization or repository name have uppercase letters HOT 1
- Unity 6 HOT 1
- Support Private Package Registries in Project testing
- Print to console
- `Warning: Not able to find annotation point for failed test!` HOT 1
- Cannot find annotation point for `Debug.LogError` stack trace
- unityLicensingServer is ignored on Windows
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from unity-test-runner.