juliapackaging / binarybuilder2.jl Goto Github PK

CUDA_Driver_jll currently abuses the JLL init hook to not only execute code at load time, but also define auxiliary functionality: https://github.com/JuliaPackaging/Yggdrasil/blob/a3649ece56eb711f060ade7aa07eaa65a315ed23/C/CUDA/CUDA_Driver/init.jl#L34-L58

However, the CUDA_Driver_jll is also used to provide a library that isn't available for all platforms. Because of the JLL init hook currently only being executed when an artifact is available, JuliaPackaging/JLLWrappers.jl#50, this currently necessitates duplicating the aforementioned auxiliary functionality for when the JLL is not available: JuliaPackaging/Yggdrasil#5751

Given the new design of the on_load_callback, which is hard to generalize to the case when an artifact is not available, it may be better to provide a toplevel_code setting or so that allows users to inject code into the JLL package that will be available at all time, i.e., also when no artifact is available. This would allow us to reliably add functionality to JLLs.

We should think clearly about how we want to specify dependencies. I believe there are three important axes for dependency specification:

1. PkgSpec -- We need to specify what to install, obviously. This could just be a name, like "Glibc_jll", or a full PkgSpec to install a particular version or URL/branch.
2. Platform -- We need to specify whether this is for the target or the host.
3. Packaging links -- We need to specify whether this dependency is a dependency of the packaged JLL or not.

In BB1, (2) and (3) are determined by using Dependency()/HostDependency() or BuildDependency()/HostBuildDependency(). I am not a huge fan of this because it feels rather messy to be constantly filtering/unwrapping dependency object types, and also because I want to be able to tackle horrendously complicated builds like a Canadian Cross in BB2. In such a case, there are three separate platforms at play, the host of the current machine, the host of the machine that will eventually run our compiler, and the host that will run the output of the generated compiler. So I would like a fully-generic solution that does not have special Host types.

How BB2 works right now

Right now, there is a JLLSource(pkgspec, platform) method that takes in a platform object as well, so it is fully-specified from the get-go. (example) BB2's BuildConfig takes in two separate lists of JLL sources, one called target_dependencies and one called host_dependencies. This is already not what I want w.r.t. the ability to do a canadian cross.

Also, most users won't use the BuildConfig API, they'll use build_tarballs() which has convenience methods like JLLSource(pkgspec) that will automatically cycle through the list of platforms passed in to build_tarballs() so as to be target dependencies, and automatically get included in JLL output.