Explain why the code is not generated in the `generate!()` macro about cargo-component

[peterhuene]

The development of cargo-component has gone through a few designs now relating to generating bindings.

The initial design was that cargo-component would generate a behind-the-scenes bindings crate that would be magically linked against as a dependency of the user's component project so that bindings would be seen as a normal crate of the generated code. To make that work, cargo-component directly depended on the cargo crate, which is a very large dependency as it make cargo-component effectively a copy of cargo with some component model bits bolted on.

It worked that way until support for resources was implemented in wit-bindgen; at that point the generated bindings needed to explicitly know the Rust names of implementing types, which is why the wit_bindgen::generate! macro takes arguments to name them. Taking inspiration from wit_bindgen::generate!, a cargo_component_bindings::generate! macro was implemented to accept these arguments and pass it under the covers to wit-bindgen; however, this macro's implementation differed from wit_bindgen::generate! in that it read no WIT source files; instead it would "deserialize" a resolved WIT document from a tiny wasm file generated by cargo-component. This allowed cargo-component itself complete control over the target world.

While this new design allowed cargo-component to spawn cargo and break its dependency on the cargo crate to both greatly reduce its build time and executable size, it had the downside of increasing the time it takes to build a user's component as it pushed the wit-bindgen dependency onto the user's package. It also made CI significantly slower, as it spends a lot of time building components.

To restore the component build-time impressions of users, I decided to put the code generation fully back in cargo-component; this sped up build times and CI, but turned the generate! macro into a filthy liar as it was ultimately reduced to simply including a source file generated by cargo-component. I choose not to rename the macro at the time as there was enough recent breakage from introducing it in the first place.

Ultimately this leaves us with the question "why not simply use wit_bindgen::generate! directly or wrap it with ?" if it means reducing the complexity of bindings generation and making a happier rust-analyzer by default, at the cost of increased component build times?

The answer is that cargo-component is designed to be a component-registry-first tool, similar to how cargo is a crates.io-first tool. Users should be able to easily target a world even seeing a WIT file and they should be able to consume interfaces from WASI and elsewhere in their custom WIT definitions without copying dependency files around like the Internet isn't a thing.

That support exists in cargo-component today; the elephant in the room is that an operational registry service it can talk to does not. This is something being actively worked on, so hopefully this will all make sense from a DX perspective in the near (but quite long overdue) future.

[peterhuene]

That said, I'm happy to take a breaking change to rename the macro to something more suitable.

A better approach, I think, is to probably stop hiding behind magic and remove the macro entirely.

If instead we simply generated the bindings into src/bindings.rs (possibly with a destination override in Cargo.toml) with a big "this is a generated file" warning on top and had cargo component new replace the macro invocation with a mod bindings, then it would probably solve a lot of problems.

[peterhuene]

Removing the macro proved to be relatively straight forward to implement.

I'll put up a PR to see if we want to include the (very much breaking) change with the imminent cargo-component release.

[tomasol]

@peterhuene Thanks for the clear explanation. I have migrated my project to v0.7.0 to understand the change better. Here are my observations:

• generated bindings add some noise to PRs, although this can be mitigated using .gitattributes
• if not all functions are used, they trigger a cargo warning. This can be fixed by prepending #![allow(dead_code)] to each file
• after I've fixed it in my fork, bindings were not regenerated. This is because nix sets mtime of the exe to the start of Unix epoch.

There is a concern that using a registry would cause delays.. But even in that case, I would expect (same as with cargo) to have the wit files downloaded once and stored in a local cache.

I haven't seen generating sources in other Rust projects, but one nice thing is that the auto-complete finally works 🎉 . Hopefully rust-analyzer will be fixed soon. Perhaps in the future users could be able to choose between the proc-macro and the generated sources.

BTW, how are the resources configured now that the macro is gone? I haven't used this feature yet.

[peterhuene]

if not all functions are used, they trigger a cargo warning. This can be fixed by prepending #![allow(dead_code)] to each file

One can append #[allow(dead_code)] to mod bindings too, but I decided not to as I think we should be treating any dead code warnings coming from the bindings as bugs (either the emitted code could have the proper allows or dead code shouldn't be emitted).

after I've fixed it in my fork, bindings were not regenerated. This is because nix sets mtime of the exe to the start of Unix epoch.

That's unfortunate (but I'm sure there's a good reason for doing that). A workaround is to just touch Cargo.toml for the same effect, but perhaps we should keep track of what version of cargo-component generated the bindings and regenerate it if they differ, rather than using the executable mtime.

There is a concern that using a registry would cause delays.. But even in that case, I would expect (same as with cargo) to have the wit files downloaded once and stored in a local cache.

For what it's worth, the registry client implementation does cache packages locally, but it doesn't use .wit files; it uses .wasm files (encoded from wit source files) instead.

BTW, how are the resources configured now that the macro is gone? I haven't used this feature yet.

cargo-component picks a default type name for any resources, which can be overridden by settings under the [package.metadata.component.bindings] table in Cargo.toml.