'make test' fails for julia-1.11.0-beta1 about julia

[fatteneder]

IIUC the logs correctly, then in make_log.2.log the loading.jl tests all passed, because there appear two loading.jl entries in that file, once with started and once with elapsed time.
However, loading.jl fails in make_log.1.log (at the relocatable upgrades #51989 test, also failed here #54234 (comment)) , so there seems to be some randomness involved, but nothing obvious as to why that particular test should fail.

[sergstesh]

there seems to be some randomness involved

A typical phenomenon in multi-threaded environment. In the best case it's too short timeout values, otherwise algorithmic error.

...

And don't miss the dangling Julia process - looks like the kill signal hasn't been properly sent.

...

If one decides to reproduce the tests, make sure there are, say, 40GB virtual memory available (my swap file is 64GB, I saw more than 32GB swap used during the tests execution, but I have a web browser with a lot of windows/tbs open, and it consumes, say, 10GB.

...

Regarding test failures - pay special attention to

    858 loading                                          (3) |         failed at 2024-04-26T23:13:41.669
    859 Test Failed at /media/sergei/4c7aa17d-44cf-423c-b211-ce583883925c/home/sergei/1/julia-1.11.0-beta1/test/loading.jl:1365
    860   Expression: success(addenv(`$(Base.julia_cmd()) --project=$project_path --startup-file=no -e 'import Pkg; Pkg.develop("Foo51989"); Pkg.precompile(); exit(0)'`, "JULIA_DEPOT_PATH" =>    860  depot))

in 'make_test.1.log' and to

   1002 cmdlineargs                                      (5) |         failed at 2024-04-27T00:19:48.212
   1003 Test Failed at /media/sergei/4c7aa17d-44cf-423c-b211-ce583883925c/home/sergei/2/julia-1.11.0-beta1/test/cmdlineargs.jl:1150
   1004   Expression: success(pipeline(setenv(`$(Base.julia_cmd()) --bug-report=rr-local -e 'exit()'`, "JULIA_RR_RECORD_ARGS" => "-n --nested=ignore", "_RR_TRACE_DIR" => temp_trace_dir); ))

,
i.e. in 'make_test.1.log' the failing test is 'loading.jl' and in 'make_test.2.log' the failing test is 'cmdlineargs.jl' - if I understand the diagnostic messages correctly.

[fatteneder]

Yes, I noticed that too that both failures happen when the tests are hanging inside a separately spawned Julia process, here and here.

[fatteneder]

When I just run this part here locally then it also takes ~2 min. The reason it takes so long is that that test sets up an empty depot without any caches, and so I think the spawned julia process then first has to re-compile all of Pkg.jl and its dependencies before the actual script for that instance can run.

If that's true, then I guess that's fixable.
But I am curious as to why that problem doesn't show in CI. Maybe the CI machines are just beefier?

[sergstesh]

Maybe the CI machines are just beefier?

What is "CI" ?

[fatteneder]

What is "CI" ?

continuous integration, the automated testing pipeline