Adding support for new languages about buck2

[steveklabnik]

not sure where else to ask questions? Sorry.

There's the Discussions tab as well, but in practice the team doesn't seem to enforce a distinction much, so you should be good here too :)

(I don't have good answers for your other questions, but figured I'd at least mention this.)

[zjturner]

I'm telling you what that missing piece of glue is 😃 It's that your jai_static_library rule doesn't return a provider that the C++ rules understand. The reason adding your cpp lib doesn't cause the jai lib to be built when you build the cpp lib is because the cpp lib doesn't know how to consume any of the providers returned by your jai lib in a way that would force the jai lib to be materialized.

For example, when I build a C++ library (e.g something that was created with cxx_library(), and I run buck2 audit providers --list //:cxx_lib, I get some output like this:

  - AndroidPackageableInfo
  - CPreprocessorInfo
  - CxxCompilationDbInfo
  - DefaultInfo
  - JavaPackagingInfo
  - LinkCommandDebugOutputInfo
  - LinkExecutionPreferenceInfo
  - LinkGroupLibInfo
  - LinkableGraph
  - LinkableRootInfo
  - MergedLinkInfo
  - ResourceGraphInfo
  - ResourceInfo
  - SharedLibraryInfo
  - TemplatePlaceholderInfo
  - UnstrippedLinkOutputInfo
  - XcodeDataInfo

You can eliminate some of these as being obviously irrelevant, so the ones you need to look into are:

- LinkCommandDebugOutputInfo
- LinkExecutionPreferenceInfo
- LinkGroupLibInfo
- LinkableGraph
- LinkableRootInfo
- MergedLinkInfo
- UnstrippedLinkOutputInfo
- SharedLibraryInfo

Even some of these are probably irrelevant (I don't know the whole hierarchy of C++ providers in detail). You need to examine the cxx_library() and cxx_binary() rules implementaions and see which providers they're reading from that cause arguments to be added to the linker command line. Somewhere deep in the bowels of cxx_binary() you're going to find the place where it builds the response file for the linker. In order to do that, it's going to be examining the providers of all of its deps. One of the above providers is going to be the magic one that cxx_binary() uses to decide "add foo.lib to the linker command line". That's the provider you need to return from jai_static_library. If you return that provider from jai_static_library in a way that is correctly filled out, it will work.

One thing you have to undestand about buck2 is that it's lazy to the maximum extent possible. Adding a dep from A to B does nothing by itself. All it does is pass information about B into A. If A consumes this information, that's when it will cause work to happen in B.

Imagine a hypothetical rule like jai_static_libraries that somehow magically output 2 static libraries. You make a target called two_jai_libraries (e.g. jai_static_libraries(name = "two_jai_libraries", ...) And then another rule like jai_binary that took your two_jai_libraries as a dep.

Depending on what jai_binary does, your jai_static_libraries rule might build 0, 1, or both of those outputs. It all depends on which outputs the consuming target actually consumes.

[zjturner]

The builtin C++ rules understand the builtin C++ providers. If your jai_static_library rule returns the proper builtin C++ provider object, the linking should happen automatically. Constructing the proper provider is probably not straightforward. If I were trying to do this, I would probably start by making a prebuilt_cxx_library() as a test, and then using buck2 audit providers to see what providers it returns. See if it's easy to create the same thing from your jai_static_library rule. It probably isn't, but at least you have a starting point.

Next, I'd probably start looking in the cxx prelude rules (especially prebuilt_cxx_library()) to see if it uses some internal helper functions that maybe you could piggyback off of and just call to do the heavy lifting for you.

Another idea which is something I've been meaning to try but haven't found the time yet, is to use anon rules for this. Using anon rules you can invoke a rule from within another rule. So maybe there's a way your jai_static_library could set up the necessary actions and providers to build your .lib, but then invoke prebuilt_cxx_library() as an anon rule, passing in your static lib as an input, and then returning from your own rule the same provider that prebuilt_cxx_library() returned. This way cxx_binary() would see it and magically do the right thing.

All of these are just ideas though, unsure if they'll work in practice.

[forrestthewoods]

Hrm. I've tried poking around the Rust and Go preludes. They both references cxx quite a bit.

I feel like this should be simple. I can successfully run buck2 build //jai/modules:my_jai_lib and it produces my_jai_lib.lib. I can add this as a dep in //cpp/BUCK. Interestingly this doesn't cause the Jai lib to actually be built. And if I manually build the Jai lib it obviously doesn't get linked.

So close. There's just some small piece of glue that's missing. But I can't tell what it is. And afaict there is no "step debugger" for rules. So I can't step through Starlark code and figure anything out.

[forrestthewoods]

I think I maybe need a MergedLinkInfo. But I'm struggling to figure out how to declare out as a LinkInfo. My jai.bzl is brain dead simple right now. Ignore the hardcoded paths. Easy to fix those once it's working for real.

def jai_static_library_impl(ctx: AnalysisContext) -> list[Provider]:
    out = ctx.actions.declare_output("my_jai_lib.lib")

    # this successfully produces my_jai_lib.lib
    cmd = cmd_args(["C:/Stuff/jai/jai-beta-1-089/jai/bin/jai.exe", "C:/source_control/fts_buck_jai/jai/build.jai", "-", out.as_output()])
    ctx.actions.run(cmd, category="jai")

    return [
        DefaultInfo(default_output = out),

        # what provider will cause my_jai_lib.lib to be linked downstream? and how do we create said provider

        # this is wrong and doesn't even make sense. simple case has no transitive deps. but this line appears in numerous similar places
        create_merged_link_info_for_propagation(ctx, filter(None, [d.get(MergedLinkInfo) for d in ctx.attrs.deps])),
    ]

jai_static_library = rule(impl = jai_static_library_impl, attrs = {
    "out": attrs.string()
})