Rethink how we use Preferences about snoopcompile.jl

[timholy]

I'm reasonably sure it can happen (we did get segfaults before we introduced this feature when people were rolling their own). I think it can come from the following combination of factors:

• we store multiple cache files for each package
• you can load packages from a different environment (both "moving up" the chain of environments, or by switching environments)
• with an environment variable, there's no record of whether the cache file was created with or without precompilation, and so the precompilation state cannot be used to check for consistency
• once we load a package, all future packages that depend on that package will use the in-memory version.

I think this is all you need to have bad things happen in practice: say A gets used by B & C; B gets precompiled without A being precompiled, but C gets precompiled with A being precompiled. If you load B before C, boom.

[timholy]

Hmm, you're right that as more and more packages use it, the "escape" I thought we had in https://discourse.julialang.org/t/psa-for-snoopprecompile-turning-off-extra-workload-for-specific-packages/92865/5?u=tim.holy doesn't really scale.

We should change this while also moving SnoopPrecompile to JuliaLang and possibly renaming it.

[serenity4]

With a package preference, you would still need to re-precompile a package, even though the intention was to specify that you don't want it to run the precompilation workload in future invalidations. To avoid this, we could use an environment variable instead, e.g. JULIA_SKIP_SNOOPED_PRECOMPILE=PackageA,PackageB with a coma-separated list of packages. One downside however is that re-enabling the run of precompilation workloads for a package will not cause the package to re-precompile. Would we have a way to manually invalidate a package and retrigger precompilation of affected packages in a project?

[timholy]

The whole point of using Preferences is to avoid the myriad problems that come from using an environment variable. In the worst case you can get segfaults if the compilation status of the system is inconsistent.

[serenity4]

I see the value of Preferences in the general case, but here might we not assume that a package's logic should be unaffected by whether the precompilation workload is run or not? From my perspective, the related code would only control the amount of things that end up in the package image. That is based on the assumption that no methods, types or globals are defined in the @precompile_all_calls block. Under this assumption, are the potential issues you mention still likely to happen?

Whether or not we'd like to make such an assumption would be another question.

In any case, having a preference per-package would be a clear improvement over the status quo.

[timholy]

Starting with 1.9, precompilation & package loading basically work like a (very flexible) C linker. That's how we handle the native code. If you expect to be able to link against things that are already precompiled (because they were available when you compiled the package), and then those objects disappear behind your back, bad things happen. Consistency across the entire set of dependencies is paramount, and there's no way to guarantee consistency if environment variables control what does and does not get precompiled.

[serenity4]

I see, though I wonder if that situation would happen in practice. The only way I see is if the build of a whole project somehow relies on two independent builds of that same package, which produce different images; but otherwise, wouldn't the state of the compiled code for a given package be read directly from its package image after precompilation? Given that the effect of such an environment variable would always be tied to an invalidation of the whole chain of dependency downstream, I do not see how objects would be able to vanish without triggering recompilation of dependent code.

[timholy]

Fixed by PrecompileTools