Comments (43)
just found solution in my case, just changing uses: game-ci/unity-test-runner@v2
to uses: game-ci/[email protected]
maybe there some run test code in new game-ci version that I can't used in game-ci test runner
from unity-test-runner.
I guess this could be interesting: https://docs.unity3d.com/2022.3/Documentation/Manual/CLIBatchmodeCoroutines.html
TLDR: Not all Coroutines are supported in batch mode, most notably: WaitForEndOfFrame . Which might be part of this issue for some.
from unity-test-runner.
Probably related https://answers.unity.com/questions/1888121/unity-gets-stuck-on-applicationreload-when-enterin.html
from unity-test-runner.
Looks like one of the releases caused this. It would be extremely helpful if you could help us find out which one.
Do the versions
v2.0.1
andv2.0.2
work for you too?
I have tested v2.0.1
and v2.0.2
. Both of these versions get stuck in the "Testing in playmode" phase
from unity-test-runner.
No problem. I am not sure what is wrong. The maintainer should answer this question. FIY, I have tested v2.0.0
, v2.0.1
, and v2.0.2
but only v2.0.0
works as intended
from unity-test-runner.
@ugnelis
Yeah, I'm also using game-ci for a while and I'm super happy with it.
@asyasmi what output do you get?
Creating "/github/workspace/ubuntu-latest_2021.3.21f1_StandaloneLinux64_artifacts" if it does not exist.
###########################
# Project directory #
###########################
total 56K
drwxr-xr-x 11 1001 123 4.0K May 19 10:54 .
drwxr-xr-x 4 root root 4.0K May 19 10:54 ..
drwxr-xr-x 2 1001 123 4.0K May 19 10:52 .ci
drwxr-xr-x 8 1001 123 4.0K May 19 10:52 .git
-rw-r--r-- 1 1001 123 210 May 19 10:52 .gitattributes
drwxr-xr-x 3 1001 123 4.0K May 19 10:52 .github
-rw-r--r-- 1 1001 123 1.2K May 19 10:52 .gitignore
drwxr-xr-x 4 1001 123 4.0K May 19 10:52 Assets
drwxr-xr-x 2 1001 123 4.0K May 19 10:52 Packages
drwxr-xr-x 2 1001 123 4.0K May 19 10:52 ProjectSettings
-rw-r--r-- 1 1001 123 51 May 19 10:52 README.md
drwxr-xr-x 2 root root 4.0K May 19 10:54 _activate-license
drwxr-xr-x 2 root root 4.0K May 19 10:54 ubuntu-latest_2021.3.21f1_StandaloneLinux64_artifacts
###########################
# Testing in editmode #
###########################
that's all before the cancellation of the job :(
Regarding timeout-minutes
- we had no timeout at all, it still never for out of this state.
But I think this time I got it.
UPD: Eventually that were our tests.
The issue was with tests, one of them eventually was broken and just hung a process and logs were not shown before they were done and tests were never done, so... The job ended in this limbo.
So I'd really be waiting for game-ci/cli#12 to be realized.
from unity-test-runner.
I have messed with same behaviour... Test runner just freezing on "Testing in playmode"
from unity-test-runner.
Same problem here. Locally the tests take around 7s to complete.
I canceled the test action after around 20min.
Is there any way to get more information on whats going on after "Testing in Playmode" is logged?
from unity-test-runner.
We're getting the same problem with both edit and playmode tests.
from unity-test-runner.
Any update with this problem? I've got the same issue after update unity 2020.3.38f1
from unity-test-runner.
Any update with this problem?
Updates would be posted in this thread. It doesn't look like a complete failure for projects and all versions, since there's only 5 reports here in 2 to 3 months.
What would help is if everyone added their unity versions, and ideally also some stats on memory usage (you could use atop or other measuring tools).
Either way it may be hard to debug the cause here because the Unity Editor isn't giving us much to work with. Perhaps some kind of debug output would also help. Not sure if anyone managed to get more output than nothing at all?
from unity-test-runner.
We experienced failures using unity 2019.4.40f1
For now a workaround, we are circumventing the test-runner action and executing something like the following, which works:
- name: Run Tests
id: tests
env:
UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
run: |
docker run --gpus all --rm \
--env UNITY_LICENSE \
--volume="${PWD}:/opt/project" \
unityci/editor:ubuntu-2019.4.40f1-linux-il2cpp-1 bash -c /opt/project/ci.sh
ci.sh
#!/bin/bash
FILE_PATH=UnityLicenseFile.ulf
echo "$UNITY_LICENSE" | tr -d '\r' >$FILE_PATH
unity-editor \
-batchmode \
-nographics \
-logFile /dev/stdout \
-quit \
-manualLicenseFile $FILE_PATH
platforms=(editmode playmode)
COVERAGE_OPTIONS="pathFilters:+/**/Assets/Scripts/**;assemblyFilters:+assets;generateAdditionalMetrics;generateHtmlReport;generateBadgeReport"
for platform in "${platforms[@]}"; do
unity-editor \
-runTests \
-projectPath /opt/project \
-testResults /opt/project/Logs/${platform}_test_results.xml \
-logfile /dev/stdout \
-debugCodeOptimization \
-enableCodeCoverage \
-testPlatform $platform \
-coverageResultsPath /opt/project/CodeCoverage \
-coverageOptions $COVERAGE_OPTIONS
done
from unity-test-runner.
this solves our issue
from unity-test-runner.
uses: game-ci/[email protected]
That worked, thanks!
from unity-test-runner.
Looks like one of the releases caused this. It would be extremely helpful if you could help us find out which one.
Do the versions v2.0.1
and v2.0.2
work for you too?
from unity-test-runner.
I am using Unity version 2019.4.28f1.
The project does not have any playmode tests only editmode tests.
[email protected] it hangs at "Testing in Editmode"
test-runner@v2 it hangs at "Testing in Playmode"
[email protected] it hangs at "Testing in Playmode"
Trying to run the tests on window-latests also failed:
Run game-ci/unity-test-runner@v2
D:\a\_actions\game-ci\unity-test-runner\v2\dist\main.js:1
index.js
^
ReferenceError: index is not defined
at Object.<anonymous> (D:\a\_actions\game-ci\unity-test-runner\v2\dist\main.js:1:1)
at Module._compile (node:internal/modules/cjs/loader:1101:14)
at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
at Module.load (node:internal/modules/cjs/loader:981:32)
at Function.Module._load (node:internal/modules/cjs/loader:822:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
at node:internal/main/run_main_module:17:47
Looks like the simlink is broken?
from unity-test-runner.
I am using Unity version 2019.4.28f1.
The project does not have any playmode tests only editmode tests.
[email protected] it hangs at "Testing in Editmode" test-runner@v2 it hangs at "Testing in Playmode" [email protected] it hangs at "Testing in Playmode"
Trying to run the tests on window-latests also failed:
Run game-ci/unity-test-runner@v2 D:\a\_actions\game-ci\unity-test-runner\v2\dist\main.js:1 index.js ^ ReferenceError: index is not defined at Object.<anonymous> (D:\a\_actions\game-ci\unity-test-runner\v2\dist\main.js:1:1) at Module._compile (node:internal/modules/cjs/loader:1101:14) at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10) at Module.load (node:internal/modules/cjs/loader:981:32) at Function.Module._load (node:internal/modules/cjs/loader:822:12) at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) at node:internal/main/run_main_module:17:47
Looks like the simlink is broken?
The windows issue is related to this issue it seems.
from unity-test-runner.
I also have this problem with play-mode tests. I tried the "2.0.0" solution but it doesn't work for me.
It would also be helpful to flush log messages as they happen instead of dumping them all at the end. Even before this bug the entire bunch of tests log messages would be dumped at one time point--after all the tests were complete--and all having almost the same time stamp.
from unity-test-runner.
Grouping and flushing output is tracked in game-ci/cli#12
Contributions are welcomed :)
from unity-test-runner.
I just tried running the test on 2021.3.14f1 and instead of getting stuck on "Testing in Playmode" it fails when trying to fetch private git packages. I wonder if that might always be the reason it fails and just does not handle it properly in earlier versions.
@ugnelis it seems like you are using private git packages too as there is a "gitPrivateToken" in your config?
from unity-test-runner.
I just tried running the test on 2021.3.14f1 and instead of getting stuck on "Testing in Playmode" it fails when trying to fetch private git packages. I wonder if that might always be the reason it fails and just does not handle it properly in earlier versions. @ugnelis it seems like you are using private git packages too as there is a "gitPrivateToken" in your config?
I still have the bug with Unity 2021.3.14f1.
It sounds to me that your fetch failing should not have anything to do with the Unity version.
from unity-test-runner.
I just tried running the test on 2021.3.14f1 and instead of getting stuck on "Testing in Playmode" it fails when trying to fetch private git packages. I wonder if that might always be the reason it fails and just does not handle it properly in earlier versions. @ugnelis it seems like you are using private git packages too as there is a "gitPrivateToken" in your config?
Probably, you need this step for running test (important line gitPrivateToken
):
# Test
- name: Run Tests
uses: game-ci/unity-test-runner@v2
id: tests
env:
UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
with:
gitPrivateToken: ${{ secrets.GIT_PRIVATE_TOKEN }}
githubToken: ${{ secrets.GITHUB_TOKEN }}
or:
# Build
- uses: game-ci/unity-builder@v2
env:
UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
with:
gitPrivateToken: ${{ secrets.GIT_PRIVATE_TOKEN }}
targetPlatform: ${{ matrix.targetPlatform }}
buildsPath: 'Builds'
buildName: ${{ needs.setup.outputs.completeName }}
Of course, you need to add Github secret token which has access of private repos (https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
from unity-test-runner.
I just tried running the test on 2021.3.14f1 and instead of getting stuck on "Testing in Playmode" it fails when trying to fetch private git packages. I wonder if that might always be the reason it fails and just does not handle it properly in earlier versions. @ugnelis it seems like you are using private git packages too as there is a "gitPrivateToken" in your config?
I still have the bug with Unity 2021.3.14f1.
It sounds to me that your fetch failing should not have anything to do with the Unity version.
Tested yesterday game-ci/unity-builder@v2
and game-ci/[email protected]
, everything works on the Unity version 2020.3.19f1
.
from unity-test-runner.
Probably, you need this step for running test (important line
gitPrivateToken
):
I do have the gitPrivateToken. I am using the same config that works for the builder:
# Test
- name: Test 🔬
uses: game-ci/unity-test-runner@v2
env:
UNITY_EMAIL: ${{ secrets.UNITY_EMAIL }}
UNITY_PASSWORD: ${{ secrets.UNITY_PASSWORD }}
UNITY_SERIAL: ${{ secrets.UNITY_SERIAL }}
with:
githubToken: ${{ secrets.GITHUB_TOKEN }}
gitPrivateToken: ${{ secrets.PRIVATE_TOKEN }}
customParameters: -username='${{ secrets.USERNAME }}' -password='${{ secrets.PASSWORD }}'
from unity-test-runner.
Probably, you need this step for running test (important line
gitPrivateToken
):I do have the gitPrivateToken. I am using the same config that works for the builder:
# Test - name: Test 🔬 uses: game-ci/unity-test-runner@v2 env: UNITY_EMAIL: ${{ secrets.UNITY_EMAIL }} UNITY_PASSWORD: ${{ secrets.UNITY_PASSWORD }} UNITY_SERIAL: ${{ secrets.UNITY_SERIAL }} with: githubToken: ${{ secrets.GITHUB_TOKEN }} gitPrivateToken: ${{ secrets.PRIVATE_TOKEN }} customParameters: -username='${{ secrets.USERNAME }}' -password='${{ secrets.PASSWORD }}'
If possible, please send here your log file. Also, be sure that you gave enough access for you generated Github access token
from unity-test-runner.
The log just says:
Error when executing git command. remote: Invalid username or password
For my 3 git hosted packages. But its the same token and user data that work for the build step.
from unity-test-runner.
Not sure how you are fetching git projects but in our case, we do through UPM as far as I remember. If you are fetching through git directly, this might be useful https://stackoverflow.com/a/73229045/2122457
Just in case I put my current CI configuration that might be useful:
# See: https://game.ci/docs/github/getting-started
name: CI
on:
workflow_dispatch: { }
push:
branches: [ master ]
pull_request:
types: [review_requested, ready_for_review]
branches: [ master ]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
test:
name: Unity Tests Workflow
runs-on: ubuntu-latest
timeout-minutes: 60
strategy:
matrix:
unityVersion:
- 2020.3.19f1
testMode:
# - playmode
- editmode
steps:
# Checkout (without LFS)
- name: Checkout Repository
uses: actions/checkout@v2
# Git LFS
- name: Create LFS file list
run: git lfs ls-files -l | cut -d' ' -f1 | sort > .lfs-assets-id
- name: Restore LFS cache
uses: actions/cache@v2
id: lfs-cache
with:
path: .git/lfs
key: ${{ runner.os }}-lfs-${{ hashFiles('.lfs-assets-id') }}
- name: Git LFS Pull
run: |
git lfs pull
git add .
git reset --hard
# Cache
- uses: actions/cache@v2
with:
path: Library
key: Library-${{ hashFiles('Assets/**', 'Packages/**', 'ProjectSettings/**') }}
restore-keys: |
Library-
# Test
- name: Run Tests
uses: game-ci/[email protected]
id: tests
env:
UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
with:
gitPrivateToken: ${{ secrets.GIT_PRIVATE_TOKEN }}
githubToken: ${{ secrets.GITHUB_TOKEN }}
# Test Results
- name: Upload Test Results
uses: actions/upload-artifact@v2
if: always()
with:
name: Test Restuls
path: ${{ steps.tests.outputs.artifactsPath }}
If you still experiencing some problems, If I were you, I would try to run Github Actions locally (for example, https://github.com/nektos/act) and try to debug your DevOps scenario on a local machine.
from unity-test-runner.
I talked with my friend, so in our project, we are not using other private git repos anymore, we have embedded them into Unity project. We have done that as not all team members use Git or have access to all private repos.
In your case, you need to follow game-ci
docs on how to set github private repos fetching.
from unity-test-runner.
I talked with my friend, so in our project, we are not using other private git repos anymore, we have embedded them into Unity project. We have done that as not all team members use Git or have access to all private repos.
In your case, you need to follow
game-ci
docs on how to set github private repos fetching.
Thanks for your help. As mentioned the same settings work for the Build step to get the private packages.
Running the actions locally is a good idea. I will try that out. Also to figure out the original issue.
from unity-test-runner.
I'd like to return to the error in the OP - the test runner gets stuck.
Any info I can add to help with handling this issue?
from unity-test-runner.
I'd like to return to the error in the OP - the test runner gets stuck. Any info I can add to help with handling this issue?
Have you tried uses: game-ci/[email protected]
(putting v2.0.0
instead of v2
)?
from unity-test-runner.
Yes. Unfortunately it didn't help in my case. (I wrote this above)
from unity-test-runner.
I've got the same hang with game-ci/unity-test-runner@v2
. See: https://github.com/game-ci/unity-actions/issues/168
I was able to reproduce it on the empty project with a public repository right after cache was restored. And I'm using Linux runner.
Workflow file: https://github.com/Ecilos/ecilos-unity-assets-sandbox/actions/runs/4288532708/workflow
from unity-test-runner.
@kenorb try:
- name: Set up Unity
uses: game-ci/[email protected] # <--
env:
UNITY_LICENSE: ${{ secrets.UNITY_LICENSE }}
with:
projectPath: .
githubToken: ${{ secrets.GITHUB_TOKEN }}
from unity-test-runner.
- name: Set up Unity uses: game-ci/[email protected] # <--
Thanks. Seems like it worked using v2.0.0
as per build 7529977372. I'm going to check for other repositories to make sure. Update: 2nd repo with more asset files worked fine as well (before it didn't).
Since v2.0.0 is older, do you know what's the (known) issue with the newer version?
from unity-test-runner.
I am also facing this issue with playmode tests. I am using unity version 2020.3.37f1
. I tried using v2.0.0
but it still gets stuck in the "Testing in playmode" phase.
from unity-test-runner.
Any chance your tests involve a coroutine that has yield return null;
in it? I've seen that cause it to hang sometimes.
from unity-test-runner.
Yes, there are multiple coroutines which has yield return null;
in it. Since I am new to the unity world, I don't know what will come in place of yeild return null;
.
I tried adding -nographics
and -quit
as a customParameters and the build passed successfully BUT it did not run any tests.
✅ PlayMode Test Results - 0/0 - Passed in 0.000s
from unity-test-runner.
Hey,
Suddenly started to get this
###########################
# Testing in editmode #
###########################
again, tested v2
, v2.0.0
, v2.0.1
, till v2.1.1
but with no luck :(
I use 2020.3.36f1
, 2021.3.21f1
, 2022.2.12f1
.
Somehow fixed it the last time, but here it happens again.
from unity-test-runner.
@asyasmi what output do you get?
from unity-test-runner.
To this day I am using v2.0.0
works like a charm.
Some advice from experience using this action:
- Check Github Actions
timeout-minutes
are enough for your project. In my case, we have settimeout-minutes: 60
- Check on your computer if a clean build doesn't take too long. One of the problems might be related to shader variances.
from unity-test-runner.
I'm not sure if the progress logs get printed to the Unity log, but I do this in my tests to see what test caused a hang (works in my Core tests).
public static class TestHelper
{
private static Stopwatch _stopwatch;
public static void Setup()
{
if (_stopwatch == null)
{
// Other global setup
_stopwatch = Stopwatch.StartNew();
}
// Other per-test setup
TestContext.Progress.WriteLine("Begin time: " + _stopwatch.Elapsed.ToString() + ", test: " + TestContext.CurrentContext.Test.FullName);
}
public static void Cleanup()
{
// Other per-test cleanup
TestContext.Progress.WriteLine("Success time: " + _stopwatch.Elapsed.ToString() + ", test: " + TestContext.CurrentContext.Test.FullName);
}
}
Then in all of my test classes, I put this:
[SetUp]
public void Setup()
{
TestHelper.Setup();
}
[TearDown]
public void Teardown()
{
TestHelper.Cleanup();
}
from unity-test-runner.
Thanks, @timcassell !
I'll think of implementing something similar on our side.
from unity-test-runner.
Related Issues (20)
- Action freezes when Unity Test Framework is not imported to project
- Check Run Appears Under Wrong Workflow Run When Test Runner is Run Twice on the Same Commit
- Package Testing Hangs if Package Being Tested is the Root Directory of the Repository HOT 2
- Feature: Support Windows Runners for Package Testing
- Feature: Support Package Testing for Packages with Dependencies Outside of the Unity Registry HOT 1
- Feature: Add Caching for Package Testing
- Feature: Allow `unityVersion` of `auto` for Package Testing
- Code Coverage does not write any output anywhere HOT 2
- Unity no longer supports manual activation of Personal licenses HOT 2
- Create breaking change that updates rolling version number from 1 to 2
- Inconsistent licence activation HOT 7
- Reasource not Available by Integration HOT 5
- 'Unclassified error occured while trying to activate license.' HOT 9
- cannot access '/github/workspace/CodeCoverage': No such file or directory HOT 5
- v4.1.0 causing activation to fail when using manual offline activation HOT 5
- fullProjectPath is not set when using runAsHostUser
- packageMode workflow doesn't run playmode tests HOT 2
- chmod: cannot access '/github/workspace/CodeCoverage' HOT 1
- Licensing fails inconsistently during test step HOT 1
- Feature: Add support for Nuget dependency resolution when testing packages HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from unity-test-runner.