Coder Social home page Coder Social logo

cargo-component's People

Contributors

acfoltzer avatar alexcrichton avatar cardoso avatar eduardomourar avatar elliottt avatar esoterra avatar figsoda avatar jeffparsons avatar kajacx avatar macovedj avatar morenol avatar nacardin avatar nated0g avatar peterhuene avatar ricochet avatar rylev avatar saulecabrera avatar sunfishcode avatar thomastaylor312 avatar tomasol avatar vwkd avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cargo-component's Issues

component build doesn't seems to respect local world prefix

I have a WIT definition with something like this

package infinyon:operator

interface operator-graph {
......
}

world operator-component {
  import operator-graph
  export run: func()
}

However, when I run the build (non-reactive), it always generates a component with root prefix instead of 'infinyon`

package root:component

world root {
  import infinyon:operator/operator-graph

  export run: func()
}

Is there anyway to change this behavior? or am I misunderstanding behavior of world?

Installing with `cargo install --path .` fails with weird error

This commit in the registry broke installing cargo-component: bytecodealliance/registry@6565977

Would you be open to pin the warg-* git commit to specific commits? That would help get rid of this kind of errors, and make sure that updates of cargo-component's dependencies are under control.

STR

  • cargo install --path .

Observed

error[E0432]: unresolved import `warg_crypto::hash::DynHash`
  --> src/registry.rs:27:25
   |
27 | use warg_crypto::hash::{DynHash, Sha256};
   |                         ^^^^^^^
   |                         |
   |                         no `DynHash` in `hash`
   |                         help: a similar name exists in the module: `AnyHash`

error[E0063]: missing field `head` in initializer of `PublishInfo`
   --> src/lib.rs:403:20
    |
403 |     let mut info = PublishInfo {
    |                    ^^^^^^^^^^^ missing `head`

error[E0063]: missing field `head` in initializer of `PublishInfo`
   --> src/lib.rs:524:20
    |
524 |     let mut info = PublishInfo {
    |                    ^^^^^^^^^^^ missing `head`

Some errors have detailed explanations: E0063, E0432.
For more information about an error, try `rustc --explain E0063`.
error: could not compile `cargo-component` due to 3 previous errors
error: failed to compile `cargo-component v0.1.0 (/home/ben/code/cargo-component)`, intermediate artifacts can be found at `/home/ben/code/cargo-component/target`

Feature: eliminate `wit_bindgen_rust` dependency from user projects.

Currently cargo component generates bindings as external crates and, in theory, user projects don't need a dependency on wit_bindgen_rust for code generation (only cargo-component itself does).

However, the wit_bindgen_rust crate exports the canonical ABI functions that tools like wit-component are expecting to be exported from the user's core module. The wit_bindgen_rust crate has dozens of dependencies that aren't needed at all for cargo component projects.

An alternative approach might be to additionally generate the canonical ABI exports and link them in, similar to how bindings are generated.

Expose common pass through commands in `--help`

cargo component passes through certain commands like build to be run by cargo. The blurb at the bottom of --help explains this.

Unrecognized subcommands will be passed to cargo verbatim after
relevant component bindings are updated.

However, it would be helpful to document the common commands that you might want to pass through to cargo like cargo component build. This would be more clear to new users to understand how to build their component.

General issue with 'cargo' components

This proposal is way too Rust specific and out of scope.

Having a registry for wit components will likely one day become a field of power struggle. I'm not interested in that, so in that sense having a relatively neutral Rust-like registry mechanism could be beneficial. On the other hand the word cargo and the Cargo.toml format are way too specific. Adding cargo like semantics to s-exp formatted wit would be aesthetically much more fitting.

Update README regarding WASI

We should update the README to explain why cargo component build currently targets wasm32-unknown-unknown (because a WASI preview based on the component model is still in development).

It should explain that the intention is to target WASI by default in the future.

wasi component runtime error: instance export `read` has the wrong type

After installing a the newest version of cargo component, I get this error:

Error: import `wasi:io/streams` has the wrong type

Caused by:
    0: instance export `read` has the wrong type
    1: type mismatch with results
    2: expected `bool` found `enum`

Full reproducible example can be found here. Look at the plugin-wasi for the guest code and runtime-rust-wasmtime for the host. The build-and-run-sys.sh script shows what commands I use to build the guest and run the host.

Redesign `Cargo.toml` format

Currently, we mix exported and imported interfaces in the dependencies section and that can be quite confusing.

Originally it was intended that the [package.metadata.component.dependencies] section would be 1:1 with the normal [dependencies] section.

But having to mark some of the entries as export = true is clumsy; it's additionally clumsy to treat the default exported interface as the exported interface without a version.

Instead, I propose we change the format to be:

[package.metadata.component]
# Specifies the name of an export interface to export directly from the component
default = "interface"

[package.metadata.component.imports]
foo = "foo.wit"

[package.metadata.component.exports]
interface = "interface.wit"
bar = "bar.wit"

For now I also propose we remove the version field since it's currently unused and just use paths to the interface files to make thing cleaner.

Add `test` command to support running tests

Currently there is no test command in cargo-component. The standard cargo test doesn't work because the standard tooling has no visibility to bindings, exports or any generated structs and enums. I'd think it's a good devX that devs can follow the same workflow just like when writing regular Rust apps. Please point me to the right direction if I miss any docs wrt testing!

Build broken by dependency updates in wit-bindgen

I finally got around to trying out this nice project, but got stuk when trying to build a vanilla "demo" project.

cargo component new demo
cd demo
cargo component build

Although my Rust skills are very minimal, I succeeded in getting the build running by:

  1. Cloning the cargo component
  2. Modifying the Cargo.toml to depend on the commit just before the breaking one that updates a.o. the with parser version: bytecodealliance/wit-bindgen@86401dc
  3. Installing this local version instead of the latest Git version.
  4. Modifying the Cargo.toml file of the demo project to depend on the cargo-component-bindings crate on my local machine.

I haven't yet gotten further than building the demo project, so I'm not sure if there are other issues of course.

Hopefully this helps a little in some way, and I'm looking forward to playing more with this project and WASM Components in general. I find the work very impressive!

Feature: implement a `cargo component add` command.

Now that cargo-add has been merged into cargo proper, we should definitely have an analogous command for cargo-component.

Something as simple as cargo component add --version 0.1.0 foo.wit editing Cargo.toml to include:

foo = { version = "0.1.0", path = "foo.wit" }

And cargo component add --version 0.1.0 --export foo.wit adds:

foo = { version = "0.1.0", path = "foo.wit", export = true }

error: module does not export a function named `cabi_realloc`

I recently updated cargo component and while trying to compile a component, I got the following error while running cargo component build -p template-barebones --target wasm32-unknown-unknown --release

...
   Compiling wit-bindgen v0.7.0
   Compiling ec-core v0.1.0 (/Users/jake/codebases/entropy/constraints/core)
   Compiling template-barebones v0.1.0 (/Users/jake/codebases/entropy/constraints/examples/barebones)
    Finished release [optimized] target(s) in 9.19s
    Creating component /Users/jake/codebases/entropy/constraints/target/wasm32-unknown-unknown/release/template_barebones.wasm
error: module does not export a function named `cabi_realloc`

That said, the generated .wasm file still appears to be somewhat of a valid component, as executing wasm-tools component wit on it shows a wit (although with root for the package and world name, but that might be an unrelated issue).

With this error, the generated bytecode does not run in the wasmtime instance like it used to. Note, I'm not using WASI

I tried doing a bit of research into cabi_realloc (such as this issue) but haven't quite grasped what I did to cause this to start popping up.

Any thoughts on how to get cabi_realloc to generate? Possibly related, I'm using wit-bindgen v0.7.0, although I'm the process of updating it to v0.11.0 (I see lots of macro parameters have changed).

cargo-component.lock file created everywhere

I'm getting a cargo-component.lock file created on almost every project I open in VSCode. I'm not sure why this is happening, but I was curious if it's something that can be suppressed/disabled?

Support bindings wrapper crates that can be used in component implementation crates

This is a continuation of #75, which exposes the use case. Quoting @peterhuene here:

          For the feature request, I think it would be along the lines of:
  1. Allow component metadata in lib packages that aren't cdylib; only generate bindings.
  2. Do not attempt to componentize the output of such packages (i.e. there is no .wasm output).
  3. Ensure that linking in multiple custom sections containing wit-bindgen metadata sourced from those rlibs is properly combined when componentizing the final component (i.e. the cdylib or bin package that link the rlibs).

Originally posted by @peterhuene in #75 (comment)

[Design] Component Registry Dependencies

Component Registry Dependencies

Overview

Currently cargo-component supports implementing a component by specifying
individual imports and exports defined by local wit documents via
Cargo.toml.

At the time cargo-component was originally implemented, it was imagined that
a component registry might store individual interface definitions as packages,
enabling registry dependencies to be specified in a Cargo.toml file as:

[package.metadata.component.imports] 
pkg1 = { version = "1.2.3", package = "org/pkg1" }

[package.metadata.component.exports] 
pkg2 = { version = "1.2.3", package = "org/pkg2" }

Therefore at most one interface could be defined in a wit document and stored
in a component registry "interface" package.

As a consequence of this, a world stored in a registry would then
need to explicitly reference each interface being imported and exported from
their individual interface packages; this is certainly not ergonomic and would
definitely contribute to an unnecessary proliferation of packages.

wit has since evolved to allow multiple interfaces and worlds to be defined
in one or more wit documents. To facilitate this,
wit now has syntax for using types from other documents.

With this more flexible approach to defining interfaces, cargo-component
needs a new mechanism for specifying dependencies that are stored in a
component registry.

This document proposes a design for specifying dependencies from a component
registry for cargo-component.

Registry package types

Before discussing the proposed design, it might be useful to discuss some
terminology surrounding the types of packages that might be stored in a
component registry.

Previously, there was discussion around there being three types of packages in
a component registry: an interface package, a world package, and a
component package.

Both interface and world packages are simply WebAssembly components
containing only type information; in terms of the component model proposal,
the former describes an instance type and the latter describes a component
type.

A component package is an implementation of a WebAssembly component; thus it
contains type information and executable code.

With the introduction of the use syntax, this proposal suggests reducing the
types of registry packages to two: a wit package and the
aforementioned concept of a component package.

A wit package, much like the concepts of interface and world
packages, contains only definitions of types, interfaces, and worlds. However,
a wit package may store any number of definitions, including defining
both interfaces and worlds in the same package.

Design overview

Note: cargo-component should support sourcing packages from multiple
registries, potentially defaulting to a particular registry instance.

This design proposes that cargo-component reads only the version
requirements for component registry dependencies from Cargo.toml.

An example Cargo.toml of a "greeter" component might look like:

[package.metadata.component.dependencies]
wasi = "webassembly/wasi:1.2.3"
formatter = "my-org/formatter:3.2.1"

The short form of a dependency is <name> = "<package-id>:<version>".

The general form is <name> = { version = "<version>", package = "<package-id>" }
where name here maps to a package name used in component.wit (see below)
and package-id is the qualified identifier of the package in a component registry.

Local wit documents may still be referenced by using a path key instead of the
package key.

In this example, Cargo.toml is only specifying that version 1.2.3
of the webassembly/wasi package and version 3.2.1 of the my-org/formatter
package be used.

Note that what is used from the packages is not specified in Cargo.toml;
thus it doesn't describe the world of the component being built in
any way.

To specify the component's world, a wit file with a default name of
component.wit is used:

world my-component {
    import wasi-fs: wasi.fs
    import formatter: formatter
    greet: func(name: string)
}

In this example, the component imports two interfaces: the fs interface from
a package named wasi and the default interface from the formatter package.
The former is used to print the greeting to the console and the latter is used
to format the message based on whatever formatter implementation
is supplied at runtime.

The component will directly export a function named greet that will
ultimately print a greeting for the given name.

Because cargo-component resolves dependencies ahead-of-time, wit-parser
only needs to be instructed where to locate the wasi and formatter packages to
successfully parse component.wit.

Implementation

To implement this approach, cargo-component will parse Cargo.toml to figure
out what component registry dependencies are required.

For consistent builds, it will also consult a lock file (design TBD) for
specific versions and signatures of the dependencies to use.

cargo-component will contact one or more registries to download (or update)
the package logs of the dependencies, verify the logs, and resolve the version
requirements to specific versions to download and cache locally.

It will then parse ./component.wit (the path can be changed in Cargo.toml, if
desired) and provide wit-parser with the paths to the cached package
dependencies.

From this, the definition of a world will be derived and used to generate the
bindings needed to build the component for commands like
cargo component build.

Finally, the resulting component will encode unique urls for its imports and
exports based on what packages were resolved (and from where) by
cargo-component.

Benefits

A few benefits to this approach:

  • Dependencies are specified similar to how they are specified for Rust
    crates in Cargo.toml: version requirements go in Cargo.toml and
    what is used from the dependencies is specified in "source code"
    (component.wit in this case).

  • wit-parser does not need to be made registry-aware; packages are
    resolved to local definitions prior to its invocation.

  • Knowledge of a component.wit file is transferable to other implementation
    languages: it is just a wit document to describe the component being
    built and therefore other language-specific tooling would likely also use a
    component.wit file for generating bindings.

Tooling for other languages

In addition to Rust, this approach can also be adopted for other
language-specific tooling.

For example, in JavaScript, tooling that wraps npm could specify dependency
version requirements in package.json and use a component.wit file to
specify the component type for generating bindings.

Even wit-bindgen CLI (or a tool wrapping it) could be made registry-aware by
sourcing version requirements from a file (bindgen.toml?) and use a
component.wit file to generate bindings for supported languages.

Handle `wasi:cli/run` specially for a command component

Right now if a command component is created and a world is included via:

package example:demo 

world demo {
    include wasi:cli/command
}

then the output component will expect a Component which implements the wasi:cli/run interface. This is already implicitly created for the "command" flavor of the component and the adapter, though, so could this world be modified in this situation implicitly by cargo component to avoid the requirment that impl Guest for Component is needed to compile?

cargo-component as cargo command for rust-analyzer fails for non-component crates

After setting "cargo-component" as the custom cargo command for rust-analyzer in VS Code, it begins to fail to parse Cargo.toml's for non-cargo component crates.

"rust-analyzer.server.extraEnv": {
  "CARGO": "cargo-component"
}

Use case 1: workspaces

Cargo.toml of a workspace with only one package (repo):

[workspace]
members = [ "wasmcloud-test-util"]

An example error is:

Failed to read Cargo metadata from Cargo.toml file /Users/bhayes/repos/wasmCloud/wasmcloud-test/Cargo.toml, None: Failed to run `"cargo-component" "metadata" "--format-version" "1" "--manifest-path" "/Users/bhayes/repos/wasmCloud/wasmcloud-test/Cargo.toml" "--filter-platform" "aarch64-apple-darwin"`: `cargo metadata` exited with an error: error: manifest path `/Users/bhayes/repos/wasmCloud/wasmcloud-test/Cargo.toml` is a virtual manifest, but this command requires running against an actual package in this workspace

Running cargo metadata --format-version 1 --manifest-path /Users/bhayes/repos/wasmCloud/wasmcloud-test/Cargo.toml --filter-platform aarch64-apple-darwin succeeds. Working directory does not matter.

Use case 2: package with library

Cargo.toml of a package with a local library (repo):

Cargo metadata from Cargo.toml file /Users/bhayes/repos/wasmCloud/wash/Cargo.toml, None: Failed to run `"cargo-component" "metadata" "--format-version" "1" "--manifest-path" "/Users/bhayes/repos/wasmCloud/wash/Cargo.toml" "--filter-platform" "aarch64-apple-darwin"`: `cargo metadata` exited with an error: error: manifest `/Users/bhayes/repos/wasmCloud/wash/Cargo.toml` does not contain a `package.metadata` section

Running cargo metadata ... without "cargo-component" succeeds.

Target wasm32-wasi by default

We should switch from targeting wasm32-unknown-unknown and target wasm32-wasi by default and automatically link in an adapter for users (at least until preview2 is supported natively in the rust toolchain).

Could cargo-component auto-compose components?

Currently if I add a [package.metadata.component.dependencies] section to a component with path = '...' the final output of cargo component build has an import of an instance. Could it be possible to automatically run wasm-tools compose or the equivalent thereof to have the final component bundled in the artifact?

Naively I thought this already happened, so I figure that an option, even if off-by-default, might be good to have

How to specify the package namespace for local target dependencies

My wit documentation references the interface of a local dependency package, but [package.metadata.component.target.dependencies] requires that the package namespace must be specified

$ cargo component build
error: expected package identifier with format `<namespace>:<name>`
in `target.dependencies`

wit dir:

$  tree wit
wit
├── component.wit
└── deps
    └── test
        └── types.wit

wit/component.wit:

package component:hello-world

world command-adapter {
    import test: test.types

    export run: func() -> result
}

Cargo.toml:

[package]
name = "hello-world"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
wit-bindgen = { version = "0.6.0", default_features = false }

[lib]
crate-type = ["cdylib"]

[package.metadata.component]
package = "component:hello-world"

[package.metadata.component.target]
path = "wit"

[package.metadata.component.target.dependencies]
"test" = { path = "wit/deps/test" }

What should I set the namespace for local dependencies to?

If I specify a namespace at random, it raises a new error

[package.metadata.component.target.dependencies]
"component:test" = { path = "wit/deps/test/" }
$ cargo component build
error: failed to read content of dependency `component:test` at path `/wasm/hello-world/wit/deps/test/`

Caused by:
  Is a directory (os error 21)

Integration with rust-analyzer is broken for bindings

Repro steps

  1. cargo component new repro
  2. cd repro
  3. Run VS Code with rust-analyzer enabled and open src/main.rs.

rust-analyzer should trigger a check command that will generate a target/bindings/repro/target.wasm file containing the current target world for the project; the target is empty for command projects by default, so the generated bindings mod will be empty.

  1. Create a wit directory.
  2. Add a test.wit file with the following contents:
package repro:test

world repro {
    import test: func()
}
  1. In VS Code, manually save src/main.rs to trigger a reparse.
  2. In the main function in src/main.rs, type bindings::.

Expected results

rust-analyzer should offer autocomplete suggestions for the imported function.

Actual results

Nothing happens.

Remarks

Right now, the cargo_component_bindings::generate! macro reads the target wasm file (which is very small and only contains type information) to generate bindings and emits a include_bytes! with the hopes that a dependency on the file will trigger a rebuild if it changes.

However, it appears that rust-analyzer doesn't respect changes to the file to reparse; it continues to use a cached evaluation of the proc-macro.

This may be related to rust-lang/rust-analyzer#13668, but I can't say for sure.

Workarounds

Restarting the rust-analyzer server after modifying the WIT files forces the re-evaluation of the macro with the latest bindings information.

[Question] Export multiple components

Is there a way to export multiple components from one crate?

Use case: Implement game scripting system. Use one crate (one wasm file) for multiple game scripts instead of compile multiple crates (multiple wasm files).

use bindings::test_component;

struct Component;
struct AnotherComponent;

impl test_component::TestComponent for Component {
    fn hello_world() -> String {
        "Hello, World!".to_string()
    }
}

impl test_component::TestComponent for AnotherComponent {
    fn hello_world() -> String {
        "Another Hello, World!".to_string()
    }
}

bindings::export!(Component);
bindings::export!(AnotherComponent);
> cargo component build
   Compiling test_component v0.1.0 (C:\Dev\RustProjects\test_component)
error[E0428]: the name `__FORCE_SECTION_REF` is defined multiple times
  --> src\lib.rs:18:1
   |
18 | bindings::export!(Component);
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `__FORCE_SECTION_REF` redefined here
19 | bindings::export!(AnotherComponent);
   | ----------------------------------- previous definition of the value `__FORCE_SECTION_REF` here
   |
   = note: `__FORCE_SECTION_REF` must be defined only once in the value namespace of this module
   = note: this error originates in the macro `bindings::export` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0428]: the name `__force_section_ref` is defined multiple times
  --> src\lib.rs:18:1
   |
18 | bindings::export!(Component);
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `__force_section_ref` redefined here
19 | bindings::export!(AnotherComponent);
   | ----------------------------------- previous definition of the value `__force_section_ref` here
   |
   = note: `__force_section_ref` must be defined only once in the value namespace of this module
   = note: this error originates in the macro `bindings::export` (in Nightly builds, run with -Z macro-backtrace for more info)

Investigation: using `witgen` for default interface definitions?

Right now we create a interface.wit file to represent the default interface of the component (i.e. what the component itself exports as functions rather than named instances).

There's a really cool effort underway to use proc-macros to generate wit files from Rust types called witgen.

It would be really fantastic if Rust developers could author at least their default interface definitions using Rust types, if desired, as that would be a pretty natural fit for DX.

Notice: upcoming breaking change

What

A breaking change to how cargo-component generates bindings will be landing soon.

Manual intervention will be required to keep existing component projects building with a newer cargo-component.

When #101 lands, do the following to update your component projects:

  1. Remove the dependency on wit-bindgen: cargo rm wit-bindgen.
  2. Add a dependency on cargo-component-bindings to Cargo.toml: cargo-component-bindings = { git = "https://github.com/bytecodealliance/cargo-component" }
  3. Add a cargo_component_bindings::generate!(); to the top of your component's source file.
  4. Remove any export! macro invocations from your source.

Why

An upcoming change to wit-bindgen (see bytecodealliance/wit-bindgen#604) to support resources from the component model will preclude the manner in which cargo-component generates its bindings today.

To fully support resources, wit-bindgen will need to be provided the Rust types implementing the resources at the time of bindings generation; additionally, the generated export! macro is being removed in favor of also supplying the implementing Rust type at generation time.

Ultimately this means that cargo-component can no longer generate bindings into a separate crate and automatically inject a dependency on the bindings crate from the component crate.

However, the changes to cargo-component are not dependent on the support for resources in wit-bindgen and will likely land ahead of resources support.

How

Instead of generating bindings into a bindings crate, cargo-component will adopt a proc-macro approach similar to wit-bindgen::generate!.

The cargo-component version of this macro will differ from the wit-bindgen macro in that it will generate bindings from an encoding of the target world resulting from dependency resolution rather than parsing a local wit package.

Currently, a generated bindings crate is used for the bindings; with these changes, bindings will be generated into a bindings module in the current crate.

Here is an example of defining a component in Rust with cargo-component today:

use bindings::Example;

struct Component;

impl Example for Component {
    fn hello_world() -> String {
        "Hello, World!".to_string()
    }
}

bindings::export!(Component);

And here is what it will look like with the upcoming changes:

cargo_component_bindings::generate!();

use bindings::Example;

struct Component;

impl Example for Component {
    fn hello_world() -> String {
        "Hello, World!".to_string()
    }
}

A positive side effect of these changes is that it should improve the user experience if the project is built for the first time with cargo build instead of cargo component build: a compiler error will be emitted instructing users to build with cargo component build and a link to where it can be installed.

Additionally, as a magic crate dependency is no longer added by cargo-component, tools like rust-analyzer and cargo expand (see #91) will now work provided that their invocation follows a cargo-component command that resolves dependencies to generate bindings information.

Where

Right here, silly.

When

soon

[Question] How to strip component to normal wasm

Can't believe I hadn't seen this before! This + compose is awesome stuff!

I currently have been embedding the wit in a custom section so that a binary can be use in a 1.0 runtime. Your work on wit-bindgen to generate wit from the component is great and was wondering how hard it would be to strip the rest so that I could use the composition of components for building, but still use the final binary in an older runtime while also being able to go back to a component again.

Versioning in bindings

Currently, cargo-component injects the component version (sourced from Cargo.toml) into a locally defined target world when the package directive does not contain a version (see #111).

The intention behind this is so that Cargo.toml serves as the source of truth for the "version" of the component and, in the case of targeting a local world definition, its interfaces as well.

However, as documented in #111, this runs afoul of Wasmtime's bindgen! macro because it only supports reading local WIT files to generate the host bindings and only the package directive conveys the version of the interfaces.

The result is a mismatch between the two where the version implicitly supplied by cargo-component is not the same as the version (usually missing) used in the host bindings and the host can't load the component.

A larger question should be asked of the component model proposal: where should version information actually go in interfaces? Right now it participates as part of the extern name, which means that the version must be an identical string match even when the instance might be type compatible. This seems incorrect on the surface.

For cargo-component, the question is: how do we make this a good experience while still maintaining the goals behind versioning artifacts it produces?

One option is to stop automatically injecting the component version into the target world and let the two have independent versions so that what cargo-component and Wasmtime's bindgen! generate for bindings match; I'm in favor of doing this.

Another problem is what to do about target dependencies referenced from a registry?

For WasmCon, the hope is to publish WASI packages to a registry which can be easily targeted with cargo component new --target or used in a local world definition with cargo component add --target.

As those dependencies are definitively versioned (they must be to exist in the registry), the bindings cargo-component generates will also be versioned.

But the WIT files that were used to create the adapter modules and WASI support within Wasmtime are not versioned to match, so would not be compatible based solely on extern name.

I propose that cargo-component stops overriding the version specified in the local target definition based on the version resolved from the registry; if the version is specified in WIT (and it matches the resolved version), then it is used; otherwise if no version was specified, then there's no version used in the bindings either.

Failure to build component containing flags shape

I am using cargo-component from commit 29fea4a.

While trying to build a component that contains a flags shape (example here), the following error is presented (reproduced here):

error[E0433]: failed to resolve: use of undeclared crate or module wit_bindgen
--> src/lib.rs:1:1
|
1 | cargo_component_bindings::generate!();
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ use of undeclared crate or module wit_bindgen
|
= note: this error originates in the macro cargo_component_bindings::generate (in Nightly builds, run with -Z macro-backtrace for more info)
error[E0412]: cannot find type Permissions in this scope
--> src/lib.rs:1:1
|
1 | cargo_component_bindings::generate!();
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ not found in this scope
|
= help: consider importing this struct:
std::fs::Permissions
= note: this error originates in the macro cargo_component_bindings::generate (in Nightly builds, run with -Z macro-backtrace for more info)
error[E0433]: failed to resolve: use of undeclared type Permissions
--> src/lib.rs:1:1
|
1 | cargo_component_bindings::generate!();
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ use of undeclared type Permissions
|
= help: consider importing this struct:
std::fs::Permissions
= note: this error originates in the macro cargo_component_bindings::generate (in Nightly builds, run with -Z macro-backtrace for more info)

installation of cargo-component fails

On my Ubuntu 22.04.2 LTS
Given I want to install cargo-component

cargo install --git https://github.com/bytecodealliance/cargo-component

The build process fails

error: failed to run custom build command for `warg-transparency v0.1.0 (https://github.com/bytecodealliance/registry#ae9d8f75)`

Caused by:
  process didn't exit successfully: `/tmp/cargo-installTGpCgA/release/build/warg-transparency-eed4cddd4656d5fb/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-changed=src/proofs.proto

  --- stderr
  Error: Custom { kind: Other, error: "protoc failed: proofs.proto: This file contains proto3 optional fields, but --experimental_allow_proto3_optional was not set.\n" }
warning: build failed, waiting for other jobs to finish...
error: failed to compile `cargo-component v0.1.0 (https://github.com/bytecodealliance/cargo-component#2101df55)`, intermediate artifacts can be found at `/tmp/cargo-installTGpCgA`

Although I have the current versions of protoc and rustc:

$ protoc --version
libprotoc 3.14.0
$ rustc --version
rustc 1.68.1 (8460ca823 2023-03-20)

Thank you for any hint.

Local target up-to-date check isn't working for directories

If target is pointed at a directory (the expectation that it can be parsed as a wit package), then this breaks the up-to-date check because it is expecting a path to a file.

As a result, modifying a wit file in the directory doesn't trigger a regeneration of the bindings.

Ideally we would be able to parse the directory from the wit-parser API and get back the paths to the files it parsed so that we can include them in the timestamp checks.

Feature: implement a `clippy` subcommand.

I am certain I have messed something up in my environment but I can't seem to convince vscode and rust-analyzer to stop running cargo clippy which of course fails for the same reason that cargo check fails. Any ideas? Can I tell the world to just pretend clippy doesn't exist?

Building with `--release` fails to generate one component type section

I've got this small repro, in which if I compile with cargo-component build, the component is created and can be loaded in wasmtime; but if I do cargo-component build --release, then it fails, saying that an imported interface doesn't exist. Inspecting the produced release wasm module with wasm-tools objdump shows there's no log-world component type section, while there is one such section in the non-release build.

edit: for some reason, in the original repro I do have other libraries which follow the same pattern (it's the library that's importing a component interface), and it works for all of them. Maybe a coincidence, but it's only the logging library that doesn't work somehow.

How to describe dependencies on local packages with relative path

Hello developers,

I'm trying to understand Component Model with cargo-component. To that end I have tried to create a simple example, but I have the following error and would like to discuss it if you allow me to do it here.

$ cargo component build --release
error: failed to create a target world for package `comp2` (/workspaces/wasm-component-example/comp2/Cargo.toml)

Caused by:
    0: failed to decode component dependency `newgyu:comp1`
    1: dependency is not a WebAssembly component

cargo-component version is here.

 $ cargo component --version
cargo-component-component 0.1.0 (6af088a 2023-08-16 wasi:2750b73)

My example and what I expect

https://github.com/NewGyu/wasm-component-example

image

There are two packages, then each package has a world.

  • newgyu:comp1/random-generator
    • rand func is exported.
  • newgyu:comp2/hello
    • hello-world func is exported, and that depends on rand func that is provided by newgyu:comp1.

Directory structures

$ tree -L 2.
.
├── comp1
│   ├── Cargo.toml
│   ├── src
│   │      └── lib.rs
│   └── wit
│           └── world.wit
├── comp2
│   ├── Cargo.toml
│   ├── src
│   │      └── lib.rs
│   └── wit
│           └── world.wit

Code

comp2

comp2 is specifying a dependency on comp1 as local package reference.

[package]
name = "comp2"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
cargo-component-bindings = { git = "https://github.com/bytecodealliance/cargo-component" }

[lib]
crate-type = ["cdylib"]

[package.metadata.component]
package = "newgyu:comp2"

[package.metadata.component.target]
path = "wit"

[package.metadata.component.dependencies]
"newgyu:comp1" = { path = "../comp1/wit/world.wit" }

comp1

comp1 could be built with cargo component build --release. The followings have been built.

  • target/bindings/comp1/target.wasm
  • target/bindings/comp1/world
  • target/wasm32-wasi/release/comp1.wasm
[package]
name = "comp1"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
cargo-component-bindings = { git = "https://github.com/bytecodealliance/cargo-component" }

[lib]
crate-type = ["cdylib"]

[package.metadata.component]
package = "newgyu:comp1"

[package.metadata.component.target]
path = "wit"
world = "random-generator"

[package.metadata.component.dependencies]

Points I would like to ask about

  • Is it correct tow to describe dependencies on local packages with relative path ?
  • Is the dependency type handled as Wit when described as above ?
    • Why does into_component_world respond error?
    • Should I explicitly specify Wasm as the dependency type? How to specify that in Cargo.toml?

pub fn into_component_world(self) -> Result<(Resolve, WorldId)> {
match self {
Self::Wasm {
decoded: DecodedWasm::Component(resolve, world),
..
} => Ok((resolve, world)),
_ => bail!("dependency is not a WebAssembly component"),
}
}

Document system dependencies needed for installation

I ran into an issue when installing that I didn't have protobuf installed on my machine. It would be nice if system dependencies were documented in the README.

I imagine in addition to protobut that a clang/gcc might be necessary (due to the use of the cc crate which might be compiling some c dependencies?). Are there any others? libssl? Happy to send a PR adding this documentation to the README!

Is it possible to make a library that wraps a component implementation?

Say I have a component interface that is slightly too low-level for the data I would like to pass between my components: maybe it's a HashMap somewhere that I'd like to pass and have to pass as a Vec of pairs, because there's no concept mapping to hashmaps afaik in wit. So I would like to have a high-level library that wraps the implementation of a component interface, and then one or several Rust packages would use this library to implement the underlying component. I would like to be able to use cargo-component, for this.

I've tried putting the Cargo.toml annotations defining that the lib "implements a component when used" to my library crate's TOML, but then cargo-component can't find the output: failed to find output for component package *libhighlevel*. And it's right: there's no output, it's a lib; the real output would be the package using that library.

Is this something that's possible to do, as of today?

cargo install fails

Wanted to try some component examples, to demonstrate Wasm components, but

cargo install --git https://github.com/bytecodealliance/cargo-component

fails with the following error:

error[E0432]: unresolved import `warg_client::storage::PackageStorage`
  --> src/registry.rs:24:44
   |
24 |     storage::{ContentStorage, PackageInfo, PackageStorage},
   |                                            ^^^^^^^^^^^^^^ no `PackageStorage` in `storage`

error[E0599]: no method named `packages` found for reference `&warg_client::Client<FileSystemPackageStorage, FileSystemContentStorage>` in the current scope
   --> src/registry.rs:740:30
    |
740 |                 match client.packages().load_package(id.as_str()).await? {
    |                              ^^^^^^^^ method not found in `&Client<FileSystemPackageStorage, FileSystemContentStorage>`

It worked 1 week ago. Do I need to wait until it get's more stable?

Latest version of wasm-tools breaks cargo-component output

Wasm-tools release 1.0.35 bumped the wit-component crate to version 0.11.0 which breaks compatibility with cargo-component.

Trying to run a wasm-tools parse command on the results of cargo component build produces an error:

error: invalid leading byte (0xc) for import name (at offset 0x11)

Steps to reproduce:

  • Install wasm-tools version wasm-tools-1.0.35
  • Install cargo-component
  • Create a component with cargo component new --lib demo
  • Run cd demo; cargo build
  • Run wasm-tools parse -t target/wasm32-wasi/debug/demo.wasm
  • Observe error above
  • Install wasm-tools version wasm-tools-1.0.34
  • Run wasm-tools parse -t target/wasm32-wasi/debug/demo.wasm
  • Observe wat output of wasm file

Could the `wit` folder be included by default without any configuration?

Similar to how Cargo supports src/bin/*.rs or src/lib.rs without configuration, the idea being that a conventional structure of a project doesn't require further configuration but config is always possible. To that effect would the wit/*.wit folder be loaded by default and made available for bindings?

Required protoc version is confusing

The readme states to install a protobuf compiler of version 3.15 or greater. However, when I install protoc as instructed at http://google.github.io/proto-lens/installing-protoc.html, and afterward install cargo component, I get the following error:

error: failed to run custom build command for `warg-transparency v0.1.0 (https://github.com/bytecodealliance/registry#9d9e4b17)`

Caused by:
  process didn't exit successfully: `/tmp/cargo-installGXyW0h/release/build/warg-transparency-964a17dd4eb3e10f/build-script-build` (exit status: 101)
  --- stderr
  thread 'main' panicked at 'Building this crate requires a version of protoc (libprotoc) >=15.0, found: libprotoc 3.15.8

  Please install a suitable version of protobuf (see https://github.com/protocolbuffers/protobuf for downloads and instructions as well as https://protobuf.dev/support/version-support/ for info on protobuf's versioning schema)', cargo/git/checkouts/registry-c434769d228bc60a/9d9e4b1/crates/transparency/build.rs:57:9
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

So the error message tells me to install protoc of version >= 15.

I was able to overcome this error, by downloading a protoc binary from https://github.com/protocolbuffers/protobuf/releases and move that to my bin folder instead.

Feature: add settings.json to `cargo component new`

We should add, by default, a VS Code .vscode/settings.json to the output of cargo component new so that Rust analyzer is configured out-of-the-box.

We should also add an option to suppress the generation of the file.

`cargo-component` overwrites WIT package version with `Cargo.toml` version

cargo-component seems to automagically set a component's interface version to be that of the Cargo project rather than using the version (or lack of version specified in the WIT interface).

What works

The standard "hello, world" example that does not use an interface works as expected:

$ cargo component new --reactor example
$ cd example
$ cargo component build
$ wasm-tools print target/wasm32-wasi/debug/example.wasm | tail
(type (;13;) (func (result string)))
  (alias core export 2 "hello-world" (core func (;33;)))
  (alias core export 2 "cabi_post_hello-world" (core func (;34;)))
  (func (;14;) (type 13) (canon lift (core func 33) (memory 0) string-encoding=utf8 (post-return 34)))
  (export (;15;) "hello-world" (func 14))
  (@producers
    (processed-by "wit-component" "0.13.1")
    (processed-by "cargo-component" "0.1.0 (26e55d8 2023-08-15 wasi:2750b73)")
  )
)%

Function hello-world is exported as expected.

What does not work

However, if the WIT is updated to export the hello-world function from an interface, cargo-component automatically appends a version suffix to the interface to match that of the Cargo.toml. For example, if you change the WIT to:

package component:example

interface foo {
    hello-world: func() -> string
}

world example {
    export foo
}

And update the implementation in lib.rs to:

cargo_component_bindings::generate!();

use bindings::exports::component::example::foo::Foo;

struct Component;

impl Foo for Component {
    /// Say hello!
    fn hello_world() -> String {
        "Hello, World!".to_string()
    }
}

After rebuilding, you will see that the version 0.1.0 is attached to the interface automagically by cargo-component:

$ cargo component build
$ wasm-tools print target/wasm32-wasi/debug/example.wasm | tail
(instance (;8;) (instantiate 0
      (with "import-func-hello-world" (func 14))
    )
  )
  (export (;9;) (interface "component:example/[email protected]") (instance 8))
  (@producers
    (processed-by "wit-component" "0.13.1")
    (processed-by "cargo-component" "0.1.0 (26e55d8 2023-08-15 wasi:2750b73)")
  )
)%

After updating the version in Cargo.toml to "0.2.0" and rebuilding, the version is now 0.2.0:

$ cargo component build
$ wasm-tools print target/wasm32-wasi/debug/example.wasm | tail
(instance (;8;) (instantiate 0
      (with "import-func-hello-world" (func 14))
    )
  )
  (export (;9;) (interface "component:example/[email protected]") (instance 8))
  (@producers
    (processed-by "wit-component" "0.13.1")
    (processed-by "cargo-component" "0.1.0 (26e55d8 2023-08-15 wasi:2750b73)")
  )
)%

Even if i set the package version in the WIT package component:[email protected], the version from the Cargo.toml is set on the component interface.

This becomes problematic when writing a Rust program that uses the component, referencing the WIT (wasmtime::component::bindgen!("example" in "world.wit");). It cannot find the instance due to mismatched versioning and errors with:

Error: instance 'example:service/handler' not found

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.