dfinity / canpack

Package multiple libraries into one ICP canister.

canpack  

Canpack is a code generation tool which simplifies cross-language communication in Internet Computer canisters (such as calling a Rust crate from Motoko). This works by generating a separate canister from the host language, combining fragments defined across multiple libraries.

Note: This project is early in development; unannounced breaking changes may occur at any time.

Installation

Ensure that the following software is installed on your system:

• dfx (latest version)
• Rust >= 1.71
• Node.js >= 18

Run the following command to install the Canpack CLI on your global system path:

npm install -g canpack

Quick Start (Motoko + Rust)

Canpack has built-in support for the Mops package manager.

In your canister's mops.toml file, add a rust-dependencies section:

[rust-dependencies]
canpack-example-hello = "^0.1"
local-crate = { path = "path/to/local-crate" }

You can also specify [rust-dependencies] in a Motoko package's mops.toml file to include Rust crates in any downstream canisters.

Next, run the following command in the directory with the mops.toml and dfx.json files:

canpack

This will configure and generate a motoko_rust canister with Candid bindings for the specified dependencies. Here is a Motoko canister which uses a function defined in the canpack-example-hello crate:

import Rust "canister:motoko_rust";

actor {
    public composite query func hello(name: Text) : async Text {
        await Rust.canpack_example_hello(name)
    } 
}

Any Rust crate with Canpack compatibility can be specified as a standard Cargo.toml dependency. See the Rust Crates section for more details.

Rust Crates

It's relatively simple to add Canpack support to any IC Wasm-compatible Rust crate.

Here is the full implementation of the canpack-example-hello package:

canpack::export! {
    pub fn canpack_example_hello(name: String) -> String {
        format!("Hello, {name}!")
    }
}

If needed, you can configure the generated Candid method using a #[canpack] attribute:

canpack::export! {
    #[canpack(composite_query, rename = "canpack_example_hello")]
    pub fn hello(name: String) -> String {
        format!("Hello, {name}!")
    }
}

Note that it is possible to reference local constants, methods, etc.

const WELCOME: &str = "Welcome";

fn hello(salutation: &str, name: String) -> String {
    format!("{salutation}, {name}!")
}

canpack::export! {
    pub fn canpack_example_hello(name: String) -> String {
        hello(WELCOME, name)
    }
}

The canpack::export! shorthand requires adding canpack as a dependency in your Cargo.toml file. It's also possible to manually define Candid methods by exporting a canpack! macro:

pub fn hello(name: String) -> String {
    format!("Hello, {name}!")
}

#[macro_export]
macro_rules! canpack {
    () => {
        #[ic_cdk::query]
        #[candid::candid_method(query)]
        fn canpack_example_hello(name: String) -> String {
            $crate::hello(name)
        }
    };
}

Advanced Usage

canpack.json

Pass the -v or --verbose flag to view the resolved JSON configuration for a project:

canpack --verbose

Below is a step-by-step guide for setting up a dfx project with a canpack.json config file. The goal here is to illustrate how one could use Canpack without additional tools such as Mops, which is specific to the Motoko ecosystem.

Run dfx new my_project, selecting "Motoko" for the backend and "No frontend canister" for the frontend. Once complete, run cd my_project and open in your editor of choice.

Add a new file named canpack.json in the same directory as dfx.json.

In the canpack.json file, define a Rust canister named my_project_backend_rust:

{
    "canisters": {
        "my_project_backend_rust": {
            "type": "rust",
            "parts": [{
                "package": "canpack-example-hello",
                "version": "^0.1"
            }]
        }
    }
}

Next, run the following command in this directory to generate all necessary files:

canpack

In your dfx.json file, configure the "dependencies" for the Motoko canister:

{
    "canisters": {
        "my_project_backend": {
            "dependencies": ["my_project_backend_rust"],
            "main": "src/my_project_backend/main.mo",
            "type": "motoko"
        }
    },
}

Now you can call Rust functions from Motoko using a canister import:

import Rust "canister:my_project_backend_rust";

actor {
    public func hello(name: Text) : async Text {
        await Rust.canpack_example_hello(name)
    } 
}

Run the following commands to build and deploy the dfx project on your local machine:

dfx start --background
dfx deploy

Programmatic API

Canpack may be used as a low-level building block for package managers and other development tools.

Add the canpack dependency to your Node.js project with the following command:

npm i --save canpack

The following example JavaScript code runs Canpack in the current working directory:

import { canpack } from 'canpack';

const config = {
    verbose: true,
    canisters: {
        my_canister: {
            type: 'rust',
            parts: [{
                package: 'canpack-example-hello',
                version: '^0.1',
            }]
        }
    }
};

await canpack(config);

---

This project is early in development. Please feel free to report a bug, ask a question, or request a feature on the project's GitHub issues page.

Contributions are welcome! Please check out the contributor guidelines for more information.

canpack's Issues

The requested module '../cache.js' does not provide an export named 'isCached'

I'm getting the following error when running the canpack command

file:///Users/mike/.nvm/versions/node/v20.13.1/lib/node_modules/ic-mops/dist/commands/install.js:10
import { addCache, copyCache, isCached } from '../cache.js';
                              ^^^^^^^^
SyntaxError: The requested module '../cache.js' does not provide an export named 'isCached'

My test project can be found here. Note that I'm not even trying to import any crates yet, but even if I do I get the same error when running canpack

Is this tool in a working state right now?

My ultimate goal is to use the functions exposed in Amazon's S3 rust crate (as described in the Rust section here) from within my Motoko canister. I'm under the impression that I should be able to essentially import it into a rust canister (using canpack) and call its functions from my own Rust canister, and ultimately from one of my Motoko canisters via an inter-canister query.

If this isn't the right way to go about things, I'm all ears! I was referred to canpack via this and this forum post and the referenced thread investigating the ability to do this type of thing.

Any and all advice is appreciated, and thanks for your work on this!