Provide more insights about "Multiple Gradle daemons were used". about gradle-profiler

[ZacSweers]

We just gave up on trying to work around this and believe that Gradle's GC polling is currently flawed. We disable GC polling instead via setting the org.gradle.daemon.gc.polling.disabled system property to true and have been fine since.

[ZacSweers]

We see this somewhat regularly and the logs yield no information. We kill all daemons on the CI node before and after running the profiler as well.

* Stopping daemons
java.lang.RuntimeException: Multiple Gradle daemons were used.
	at org.gradle.profiler.GradleScenarioInvoker.checkPid(GradleScenarioInvoker.java:188)
	at org.gradle.profiler.GradleScenarioInvoker.doRun(GradleScenarioInvoker.java:117)
	at org.gradle.profiler.GradleScenarioInvoker.doRun(GradleScenarioInvoker.java:21)
	at org.gradle.profiler.ScenarioInvoker.run(ScenarioInvoker.java:17)
	at org.gradle.profiler.Main.invoke(Main.java:110)
	at org.gradle.profiler.Main.run(Main.java:76)
	at org.gradle.profiler.Main.main(Main.java:18)

* Results written to /home/ubuntu/actions-runner/_work/slack-android-ng/slack-android-ng/profile-out
  Scenario incremental_abi_app_kotlin using Gradle 6.8.2
  /home/ubuntu/actions-runner/_work/slack-android-ng/slack-android-ng/profile-out/benchmark.csv
  /home/ubuntu/actions-runner/_work/slack-android-ng/slack-android-ng/profile-out/benchmark.html

It happens after around the second warmup, and any daemon processes are coming from the profiler exec itself. Anything we could try to look for or check?

[oehme]

Make sure you're not sharing your Gradle user home between containers. By default the profiler will put it in the project directory. It contains the daemon registry and if that is shared by multiple containers running builds at the same time they will kill each others daemons.

The Gradle profiler shouldn't need more info on this, it should already be in the profile.log, because Gradle prints why it started a new daemon.

[artem-zinnatullin]

I do share Gradle user home to avoid Gradle and dependencies download on every run, but I don't run multiple containers at the same time…

And I've also seen it happening without containerization.

I'll re-investigate logs when I see it again!

[oehme]

Closing since there was no more feedback. Feel free to reopen if you have new insights.

[gabrielfeo]

Just ran into this problem myself. I imagine there's a lot of reasons a daemon can be stopped for but in my case it was a lower-than-necessary memory limit for the build.

profile.log usually mentions the cause, as @oehme said:

Daemon will be stopped at the end of the build after running out of JVM memory
Used daemon with pid 5592

The daemon logs may have more info as well: <gradle-user-home>/daemon/<gradle-version>/daemon-<pid-of-dead-daemon>.out.log (remember the default gradle-user-home profiler builds use is <project-dir>/gradle-user-home).

2022-02-04T20:55:32.968+0000 [DEBUG] [org.gradle.launcher.daemon.server.health.DaemonMemoryStatus] non-heap: GC rate: 0.00/s non-heap usage: 82%
2022-02-04T20:55:32.968+0000 [INFO] [org.gradle.launcher.daemon.server.health.LowNonHeapDaemonExpirationStrategy] Expiring Daemon due to JVM Metaspace space being exhausted
2022-02-04T20:55:32.968+0000 [DEBUG] [org.gradle.launcher.daemon.server.DaemonRegistryUpdater] Storing daemon stop event: after running out of JVM memory
2022-02-04T20:55:32.968+0000 [DEBUG] [org.gradle.launcher.daemon.registry.PersistentDaemonRegistry] Storing daemon stop event with timestamp 1644008132968

[juliocbcotta]

I am getting this too

* Stopping daemons
Running command /Users/julio.buenocotta/git/tests/gradle-user-home/wrapper/dists/gradle-7.4.2-bin/48ivgl02cpt2ed3fh9dbalvx8/gradle-7.4.2/bin/gradle --stop --gradle-user-home /Users/julio.buenocotta/git/tests/gradle-user-home
Stopping Daemon(s)
1 Daemon stopped
java.lang.RuntimeException: Multiple Gradle daemons were used.
	at org.gradle.profiler.GradleScenarioInvoker.checkPid(GradleScenarioInvoker.java:191)
	at org.gradle.profiler.GradleScenarioInvoker.run(GradleScenarioInvoker.java:154)
	at org.gradle.profiler.GradleScenarioInvoker.run(GradleScenarioInvoker.java:22)
	at org.gradle.profiler.Main.invoke(Main.java:126)
	at org.gradle.profiler.Main.run(Main.java:86)
	at org.gradle.profiler.Main.main(Main.java:25)

* Results written to /Users/julio.buenocotta/git/tests/baseline
  /Users/julio.buenocotta/git/tests/baseline/benchmark.csv
  /Users/julio.buenocotta/git/tests/baseline/benchmark.html

@ZacSweers How/where should I setup that properties? For my user? For gradle-profiler? For my project?

[kenyee]

"org.gradle.daemon.gc.polling.disabled" is a gradle property...looks like you set it via "-Porg.gradle.daemon.gc.polling.disabled=true" as a gradle param or via the GRADLE_OPTS env var

[kenyee]

I don't see anything in the checkpid code that would be affected by that flag though:
https://github.com/gradle/gradle-profiler/blob/df9ede45b5e4d21822b3bcbb38b58ef66d3deb05/src/main/java/org/gradle/profiler/GradleScenarioInvoker.java

Looks like a fairly simple pid comparison of gradle daemons...no way to opt out in that code...

[ZacSweers]

It's not a gradle property, it's a system property https://github.com/gradle/gradle/blob/32e9a659c465cea3d654512ebae8d6a5d2cc07ed/platforms/core-runtime/launcher/src/main/java/org/gradle/launcher/daemon/server/health/gc/DefaultGarbageCollectionMonitor.java#L30