Add `test` command to support running tests about cargo-component

[peterhuene]

Hi @ithinkicancode. You are correct that there currently isn't a test subcommand.

The main reason for that is that it would be nice to test the component itself in a wasm runtime rather than running it as a native executable via the default cargo test harness.

Some design needs to go into a proper implementation of that, though.

[jakehemmerle]

i've been thinking about this too. one idea is that each function under test takes a component instance as input (ie an object that exports can be accessed through) and some custom #[test] macro that takes in custom testing parameters (eg. wasi or no wasi, runtime features, etc).

syntax is wrong but something like

#[component::test(
  wasi_enable = true,
)]
fn test_export_caller(component: Component) { // or whatever the proper type is
  assert!(component.exports.call_generate_rand_nums(32).is_ok()); // calls `generate_rand_nums()`, required us to enable wasi or error otherwise
}

Currently, I've been having to do this by include_bytes! the wasm blob that's generated by cargo component build, instnatiate the blob, and then write tests around it.

[ithinkicancode]

@jakehemmerle, do you have an example of instantiating a wasm blob (from the build) and writing tests against it? I'd be interested in trying out this route. :) Would appreciate it!

[eduardomourar]

A temporary solution is to use the CARGO_TARGET_<triple>_RUNNER environment variable for that purpose. Assuming the latest wasmtime has been installed, you can run:

CARGO_TARGET_WASM32_WASI_RUNNER="wasmtime --preview2 --wasm-features=component-model" cargo test

Here is a more complete example.

[jakehemmerle]

@jakehemmerle, do you have an example of instantiating a wasm blob (from the build) and writing tests against it? I'd be interested in trying out this route. :) Would appreciate it!

@ithinkicancode

In this example, I instantiate a Wasm component with a Wasm runtime (in a wrapper) and then assert return values against its exported functions.

ec_runtime/tests/runtime.rs

/// Points to the `template-barebones` program binary.
const BAREBONES_COMPONENT_WASM: &[u8] = include_bytes!("../../target/wasm32-unknown-unknown/release/template_barebones.wasm");

use ec_runtime::{ Runtime, InitialState };

#[test]
fn test_barebones_component() {
    let mut runtime = Runtime::new();

    // The barebones example simply validates that the length of the data to be signed is greater than 10.
    let longer_than_10 = "asdfasdfasdfasdf".to_string();
    let initial_state = InitialState {
        data: longer_than_10.into_bytes(),
    };

    let res = runtime.evaluate(BAREBONES_COMPONENT_WASM, &initial_state);
    assert!(res.is_ok());
}

#[test]
fn test_barebones_component_fails_with_data_length_less_than_10() {
    let mut runtime = Runtime::new();

    // Since the barebones example verifies that the length of the data to be signed is greater than 10, this should fail.
    let shorter_than_10 = "asdf".to_string();
    let initial_state = InitialState {
        data: shorter_than_10.into_bytes(),
    };

    let res = runtime.evaluate(BAREBONES_COMPONENT_WASM, &initial_state);
    assert!(res.is_err());
}

The Wasm runtime looks like this:

ec_runtime/src/lib.rs

use wasmtime::{
    component::{bindgen, Component, Linker},
    Config, Engine, Store, Result,
};
use thiserror::Error;

/// Note, this is wasmtime's bindgen, not wit-bindgen (modules)
mod bindgen {
    use super::bindgen;

    bindgen!({
        world: "program",
        path: "../wit/application.wit"
    });
}
pub use bindgen::{ Program, Error as ProgramError, InitialState };

/// Runtime `Error` type
#[derive(Debug, Error)]
pub enum RuntimeError {
    /// Program bytecode is invalid.
    #[error("Invalid bytecode")]
    InvalidBytecode,
    /// Runtime error during execution.
    #[error("Runtime error: {0}")]
    Runtime(ProgramError)
}

/// Runtime allows for the execution of programs. Instantiate with `Runtime::new()`.
pub struct Runtime {
    engine: Engine,
    linker: Linker<()>,
    store: Store<()>,
}

impl Default for Runtime {
    fn default() -> Self {
        let mut config = Config::new();
        config.wasm_component_model(true);
        let engine = Engine::new(&config).unwrap();
        let linker = Linker::new(&engine);
        let store = Store::new(&engine, ());
        Self {
            engine,
            linker,
            store,
        }
    }
}

impl Runtime {
    pub fn new() -> Self {
        Self::default()
    }
}

impl Runtime {
    /// Evaluate a program with a given initial state.
    pub fn evaluate(&mut self, program: &[u8], initial_state: &InitialState) -> Result<(), RuntimeError> {
        let component = Component::from_binary(&self.engine, program).map_err(|_| RuntimeError::InvalidBytecode)?;

        let (bindings, _) = Program::instantiate(&mut self.store, &component, &self.linker).map_err(|_| RuntimeError::InvalidBytecode)?;

        // TODO fix this unwrap
        bindings.call_evaluate(&mut self.store, initial_state).unwrap().map_err(|e| RuntimeError::Runtime(e))
    }
}

The associated wit is:

package entropy:core

world program {
   // similar to `variant`, but no type payloads
  variant error {
    invalid-transaction-request(string),
    evaluation(string)
  }
  /// Contains signature request data that is used by the runtime. Passed into `wasmtime::Store` for state (or maybe `wasmtime::Linker`).
  export evaluate: func(config: initial-state) -> result<_, error>

  record initial-state {
    /// The preimage of the user's data under program evaulation (eg. RLP-encoded ETH transaction request).
    data: list<u8>
  }
}