Generated component has dependency on wasi:cli/ about componentize-py

[dicej]

@segeljakt Yes -- please add wasmtime-wasi to your Cargo.toml and use https://docs.rs/wasmtime-wasi/17.0.1/wasmtime_wasi/sync/fn.add_to_linker.html prior to instantiating the component. See e.g. https://github.com/bytecodealliance/wasmtime/blob/main/src/commands/run.rs for an example, including how to implement WasiView, create a WasiCtx using WasiCtxBuilder, etc.

[dicej]

@segeljakt I just posted a PR that fixes the tests: segeljakt/wasm-component-test#1

For compiling Rust, I thought it was possible to use wit-bindgen instead of cargo-component. Should I switch to cargo-component?

Yes, you can use wit-bindgen directly if you prefer, but you'll also need to provide a Preview 1->2 adapter when using wit-component to generate a component from the module, which cargo-component takes care of automatically.

I enabled async, and now it seems like I need to call Linker::instantiate_async which returns a future, instead of Linker::instantiate. Do you need an async runtime like Tokio to run WASI?

wasmtime_wasi::preview2::command::add_to_linker uses Wasmtime's async API, but you can use wasmtime_wasi::preview2::command::sync::add_to_linker instead if you want to use the synchronous API.

[kyleconroy]

Okay, I'm ready to close this out. The whole reason it wasn't working for me was that I was using wasmtime 15 in the host but wasmtime 17 to build the Python component. @segeljakt I'm glad you got your example working too.

[dicej]

Thanks for reporting this, @kyleconroy.

I think it's a duplicate of #33, and the same answer I gave there applies here. There's no built-in way to prevent componentize-py from producing components with WASI imports, but you can use WASI-Virt to convert the output of componentize-py to a component that doesn't require WASI.

[kyleconroy]

Do you have experience with WASI-Virt? I've followed the README but the component that's generated also does not work, erroring out inside get-arguments.

% wasi-virt hello.component.wasm -o virt.component.wasm

% ./target/release/wasmtime-test hello.python.wasm
Error: error while executing at wasm backtrace:
    0: 0x2136650 - <unknown>!<wasm function 24>
    1: 0x6a99 - <unknown>!<wasm function 111>
    2: 0x211aef5 - wit-component:shim!indirect-wasi:cli/[email protected]
    3: 0xa93808 - libcomponentize_py_runtime.so!componentize_py_runtime::wasi::cli::environment::get_arguments::he1e861194172045e
    4: 0xa92ef2 - libcomponentize_py_runtime.so!std::sys::pal::wasi::once::Once::call::h02a7e37fdd2f75c4
    5: 0xa8a8aa - libcomponentize_py_runtime.so!componentize-py#Dispatch
    6: 0x210f192 - libcomponentize_py_bindings.so!hello

Caused by:
    wasm trap: wasm `unreachable` instruction executed

[dicej]

In the README.md, it says:

By default the virtualization will deny all subsystems, and will panic on any attempt to use any subsystem.

So it seems to be behaving as advertised, though clearly not useful for our purposes. Ideally, it would give us an option to return an empty or statically-defined list from wasi:cli/environment/get-arguments (analogous to what it does for wasi:cli/environment/get-environment), but it doesn't appear to have that capability yet: https://github.com/bytecodealliance/WASI-Virt/blob/fd2fae04342ea58aab2426ca041da68be046b030/virtual-adapter/src/env.rs#L84.

The code causing that panic is https://github.com/bytecodealliance/componentize-py/blob/main/runtime/src/lib.rs#L414. We could potentially add an option to componentize-py to skip that, but I think it would be more appropriate to address this in WASI-Virt.

[segeljakt]

I ran into the same problem when trying to load a Python component from Rust:

const PYTHON_COMPONENT: &[u8] = include_bytes!("python-component.wasm");

fn main() -> anyhow::Result<()> {
    let engine = Engine::default();
    let mut store = Store::new(&engine, ());
    let linker = Linker::new(&engine);
    let component = Component::from_binary(&engine, &PYTHON_COMPONENT)?;
    let instance = linker.instantiate(&mut store, &component)?; // panic here
    Ok(())
}

Is there any way to configure wasmtime inside Rust to make it work?

[dicej]

@kyleconroy When I get a chance, I'll see if I can add the features to wasi-virt necessary to use it with componentize-py and add an example to the examples directory in this repo.

[segeljakt]

@dicej Hmm ok, I think I almost got it working:

use wasmtime::component::Component;
use wasmtime::component::Linker;
use wasmtime::component::ResourceTable;
use wasmtime::Config;
use wasmtime::Engine;
use wasmtime::Store;
use wasmtime_wasi::preview2::WasiCtxBuilder;
use wit_component::ComponentEncoder;

const WASI_MODULE: &[u8] = include_bytes!(concat!(
    env!("CARGO_MANIFEST_DIR"),
    "/../guest/target/wasm32-wasi/debug/guest.wasm"
));

struct Host {
    ctx: wasmtime_wasi::preview2::WasiCtx,
    table: ResourceTable,
    adapter: wasmtime_wasi::preview2::preview1::WasiPreview1Adapter,
}

impl wasmtime_wasi::preview2::WasiView for Host {
    fn table(&mut self) -> &mut ResourceTable {
        &mut self.table
    }

    fn ctx(&mut self) -> &mut wasmtime_wasi::preview2::WasiCtx {
        &mut self.ctx
    }
}

impl wasmtime_wasi::preview2::preview1::WasiPreview1View for Host {
    fn adapter(&self) -> &wasmtime_wasi::preview2::preview1::WasiPreview1Adapter {
        &self.adapter
    }

    fn adapter_mut(&mut self) -> &mut wasmtime_wasi::preview2::preview1::WasiPreview1Adapter {
        &mut self.adapter
    }
}

impl Host {
    fn new() -> Self {
        let ctx = wasmtime_wasi::preview2::WasiCtxBuilder::new().build();
        let table = ResourceTable::new();
        let adapter = wasmtime_wasi::preview2::preview1::WasiPreview1Adapter::new();
        Self {
            ctx,
            table,
            adapter,
        }
    }
}

fn main() -> anyhow::Result<()> {
    let mut config = Config::new();
    config.async_support(true);
    let engine = Engine::new(&config)?;
    let host = Host::new();
    let store = Store::new(&engine, host);
    let mut linker = Linker::new(&engine);
    wasmtime_wasi::preview2::command::add_to_linker::<Host>(&mut linker)?;

    let component = ComponentEncoder::default()
        .module(WASI_MODULE)?
        .validate(true)
        .encode()?;
    let component = Component::from_binary(&engine, &component)?;
}

When I try to run it I get:

Error: module requires an import interface named `wasi_snapshot_preview1`

I'm not sure if I'm doing something wrong when building my Guest wasm project (I use cargo build --target=wasm32-wasi), or if I should be doing something like:

    let component = ComponentEncoder::default()
        .module(WASI_MODULE)?
        .adapter("wasi_snapshot_preview1", <bytes>)
        .validate(true)
        .encode()?;

Do you have any tips?

[dicej]

@segeljakt Given that you're using cargo to build the guest, I'm not seeing how this relates to componentize-py. I'm also not clear on whether you're trying to target WASI Preview 1 or Preview 2.

If you want to target Preview 1, you'll want to build using cargo build --target=wasm32-wasi then load the resulting module using wasmtime::{Module, Linker, ...} along with https://docs.rs/wasi-common/18.0.1/wasi_common/sync/fn.add_to_linker.html. If you want to target Preview 2, you'll want to use cargo-component (which will take care of componentizing the module produced by cargo and add the Preview 1->2 adapter automatically), and use wasmtime::component::{Component, Linker, ...} along with https://docs.rs/wasmtime-wasi/18.0.1/wasmtime_wasi/preview2/command/sync/fn.add_to_linker.html. You should not need to use any part of wasmtime_wasi::preview2::preview1.

Perhaps you could explain in more detail what you're trying to do, and push what you have to a GitHub repo with steps to reproduce the issue(s) you're seeing?

[segeljakt]

@dicej Sorry I got a bit derailed. I forgot to mention that I have been trying to compile both Python and Rust to WebAssembly components. Here is a repo with the current code: https://github.com/segeljakt/wasm-component-test

The first test which targets WASM without WASI works right now, but the other two fail:

running 3 tests
Image resized and saved to cat_resized.png
test wasm_test::test_rs_guest ... ok

Error: module requires an import interface named `wasi_snapshot_preview1`
test wasm_wasi_test::test_rs_guest ... FAILED

thread 'wasm_wasi_test::test_py_guest' panicked at /Users/klasseg/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-18.0.1/src/runtime/component/linker.rs:296:9:
must use async instantiation when async support is enabled
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
test wasm_wasi_test::test_py_guest ... FAILED

For compiling Rust, I thought it was possible to use wit-bindgen instead of cargo-component. Should I switch to cargo-component?

When instantiating the Python WASI component, I first got an error:

cannot use `func_wrap_async` without enabling async support in the config

I enabled async, and now it seems like I need to call Linker::instantiate_async which returns a future, instead of Linker::instantiate. Do you need an async runtime like Tokio to run WASI?

[segeljakt]

@dicej Wow, thanks a lot! It works perfectly. I am now going to try to setup a Javascript Guest 👀

[dicej]

Yeah, if you don't mind using wasmtime-wasi (and making sure the versions line up), that's your best bet to ensure that Python's standard libraries and third party libraries work as expected, using host features like the filesystem, environment variables, random numbers, etc. as appropriate.

That said, I do think there's value in an example of using wasi-virt to virtualize some or all of the WASI features a componentize-py-generated component uses, so I still plan to add one at some point.

[kyleconroy]

If other people are looking for examples on calling the hello.component.wasm from Rust, my change to upgrade to wasmtime 17 was merged here bytecodealliance/ComponentizeJS#86