Coder Social home page Coder Social logo

zware's Introduction

zware

WebAssembly logo: a purple-blue square containing the uppercase letters W A. The square has a semicirclular notch on the top edge in the middle
Zig WebAssembly Runtime Engine

About

zware is a library for executing WebAssembly embedded in Zig programs.

Example

From examples/fib:

const std = @import("std");
const zware = @import("zware");
const Store = zware.Store;
const Module = zware.Module;
const Instance = zware.Instance;
const GeneralPurposeAllocator = std.heap.GeneralPurposeAllocator;
var gpa = GeneralPurposeAllocator(.{}){};

pub fn main() !void {
    defer _ = gpa.deinit();
    const alloc = gpa.allocator();

    const bytes = @embedFile("fib.wasm");

    var store = Store.init(alloc);
    defer store.deinit();

    var module = Module.init(alloc, bytes);
    defer module.deinit();
    try module.decode();

    var instance = Instance.init(alloc, &store, module);
    try instance.instantiate();
    defer instance.deinit();

    const n = 39;
    var in = [1]u64{n};
    var out = [1]u64{0};

    try instance.invoke("fib", in[0..], out[0..], .{});

    const result: i32 = @bitCast(@as(u32, @truncate(out[0])));
    std.debug.print("fib({}) = {}\n", .{ n, result });
}

Requirements

Compile-time

  • Zig 0.12 (master)

Run-time

  • None, zig generates static binaries:
➜  zware git:(master) ✗ ldd fib
        not a dynamic executable

Goals

  • Embed WebAssembly programs in other zig programs
  • Be fast enough to be useful

Status

  • The project is very much alpha quality
  • WebAssembly 2.0 supported (apart from the vector / SIMD support which is WIP)
  • The WebAssembly official testsuite passes (not including SIMD tests)
  • Partial WASI support

Running tests

Use zig build --help to see all the test targets, here's a summary of the important ones:

zig build test      # Run all the tests (includes unittest and testsuite)
zig build unittest  # Run the library unittests
zig build testsuite # Run all the testsuite tests
zig build test-NAME # Run the NAME testsuite test, i.e. test-type

Does it run doom?

Yes, yes it does

Screen.Recording.2023-07-17.at.02.18.40.mov

zware's People

Contributors

joshua-software-dev avatar malcolmstill avatar marler8997 avatar radarroark avatar tgschultz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

zware's Issues

Single-pass calculateContinuations

WebAssembly is designed to be parsed in a single pass. This is mostly true of foxwren. However, when calculating continuations (i.e. where to pick up after branching) we are passing back over already visited instructions.

It occurred to me that we can do a proper single pass by using a stack. I.e. when we hit a block (block, loop or if) push the offset of (or pointer to) the instruction. Then, when we hit an end pop off the top of this stack and follow the pointer / offset to update the original block instruction with the offset of the end (well, end + 1 probably)

Get rid of `findFunctionEnd`

  • This is another instance of multiple passes over opcodes where I think we can do it in a single pass.
  • Just removing the calls to that function breaks some of our test suite so we need to be careful to still error the correct errors at the correct times

Perform op stack space check on entering a function

  • The idea is rather than potentially perform stack allocation checks for instructions that produce more operands than they consume, calculate the operand stack space requirement for a whole function at parsetime and store this value with the functions metadata
  • During execution operand stack overflow checks only occur on function calls, so that instructions like i32.const don't themselves require a check
  • Additionally this feels like a move in the direction of the interpreter being a register machine rather than explicitly as stack machine

Implement testsuite

I have been implementing the testsuite tests manually in the library code.

I think:

  • I create the project with zig init-exe I need to change it to what you'd get with zig init-lib
  • I should create a testrunner program that uses the library as a dependency that can take a .wasm / .wast and execute all the tests therein
  • Set up github actions so that the testrunner runs as part of our branch tests
  • testsuite: address (#52)
  • testsuite: align (#53)
  • testsuite: binary-leb128 (#54)
  • testsuite: binary (#55)
  • testsuite: block (#56)
  • testsuite: br_if (#27)
  • testsuite: br (#28)
  • testsuite: br_table (#29)
  • testsuite: call_indirect (#32)
  • testsuite: call (#30)
  • testsuite: comments (#73)
  • testsuite: const (#21)
  • testsuite: conversions (#47)
  • testsuite: custom (#73)
  • testsuite: data (#58)
  • testsuite: elem (#59)
  • testsuite: endianness (#57)
  • testsuite: exports (#60)
  • testsuite: f32_bitwise (#42)
  • testsuite: f32_cmp (#44)
  • testsuite: f32 (#22)
  • testsuite: f64_bitwise (#43)
  • testsuite: f64_cmp (#45)
  • testsuite: f64 (#23)
  • testsuite: fac (#33)
  • testsuite: float_exprs (#61)
  • testsuite: float_literals (#62)
  • testsuite: float_memory (#51)
  • testsuite: float_misc (#50)
  • testsuite: forward (#40)
  • testsuite: func_ptrs (#75)
  • testsuite: func (#75)
  • testsuite: global (#63)
  • testsuite: if (#34)
  • testsuite: inline-module (#66)
  • testsuite: int_exprs (#67)
  • testsuite: int_literals (#68)
  • testsuite: imports (#64)
  • testsuite: labels (#41)
  • testsuite: left-to-right (#69)
  • testsuite: linking (#70)
  • testsuite: load (#71)
  • testsuite: local_get (#24)
  • testsuite: local_set (#25)
  • testsuite: local_tee (#26)
  • testsuite: loop (#33)
  • testsuite: memory (#20)
  • testsuite: memory_grow (#39)
  • testsuite: memory_redundancy (#71)
  • testsuite: memory_size (#38)
  • testsuite: memory_trap (#71)
  • testsuite: names (#71)
  • testsuite: nop (#71)
  • testsuite: return (#35)
  • testsuite: select (#71)
  • testsuite: skip-stack-guard-page (#72)
  • testsuite: stack (#37)
  • testsuite: start (#72)
  • testsuite: store (#48)
  • testsuite: switch (#36)
  • testsuite: table (#72)
  • testsuite: token (#72)
  • testsuite: traps (#49)
  • testsuite: types (#49)
  • testsuite: unreachable (#72)
  • testsuite: unreached-invalid (#72)
  • testsuite: unwind (#72)
  • testsuite: utf8-custom-section-id (#72)
  • testsuite: utf8-import-field (#72)
  • testsuite: utf8-import-module (#72)
  • testsuite: utf8-invalid-encoding (#72)

Optimisation (execution)

  • #84
  • #120
  • Fast if #132
    • Provide a faster if instruction for if blocks that have no else branch
  • Function-level stack overflow checking #130 #133
  • Reduce size of Instruction (replace usize with u32)? #134
  • Fast call
    • Provide a faster call instruction for functions internal to a module
  • Split out opcode from meta (for better cache packing)?
    • Only push metadata for instructions that have non-void metadata (better packing). Separate array for for index into the metadata array
  • Dispatch with function pointers rather than lookup table

A little help about passing string back and forth

First, thank you for this library, it works great and the user-facing code is quite easy to grasp, even for a zig and wasm newcomer as myself.

I would like to get help on how to pass strings (and therefore, serialized data structure) back and forth between the host and the module.

I understand the only way is to put the string into the memory and pass addresses and offsets to tell the module where to find the string, but I can't seem to achieve it.

I tried with Instance.getMemory() and Memory.write() but the write function only handle numbers :(
I believe I should be able to write from zware into the DATA section and then... I'm not sure and I'm lost.

Move validation entirely into `ParseIterator`

Description

Currently, validation logic is split between ParseIterator (which is what is doing the bulk of code parsing + some validation) and Validator which is doing the rest of the validation. I think these two should just be combined.

Also, ParseIterator should probably be renamed Parser and moved into its own parser.zig file. It would be great to also get rid of OpcodeIterator.

Optimise: preprocess immediates / continuations

The interpreter is slow. Make it less slow.

Low-handing fruit:

  • as a first pass there is no pre-processing pass of the instructions and therefore we are constantly reparsing LEB128s amongst other things.

Clean up

  • Naming consistency
  • Commented code
  • Code tour

Optimise execution speed

Goal

  • Speed up the interpreter. fib(39) takes around 26 seconds on my machine. This is in comparison to, say, lua on the same machine taking ~6 seconds.

Details

  • The interpreter loop is currently a large switch statement. One problem is this switch statement ends up with a single jmp instruction which won't be great for the branch predictor
  • Andrew is proposing adding support for labeled continue syntax ziglang/zig#8220, that would generate better machine code including per-instruction jmps
  • In the same issue haberman suggests using a tail-call interpreter which I'm experimenting with in #118 (which also unifies the stack...I have another branch that doesn't unify the branch and I'd like to complete both so I can compare them against one another). With the tail call interpreter I'm also experimenting with which parameters to include in the call as we'll get speed up if certain parameters can just remain in registers / avoid register spilling

References

Idea: aggressive comptime specialisation

  • Different users may have different requirements / constraints
  • Allow the consumer of the library to configure, at compile time, certain limits. E.g. if they know they will only be running small .wasm binaries, maybe they want the array of instructions to only of length less than can be represented by a u16 and thereby cut down on the size of the instructions themselves
  • More examples:
    • Customise how memory works:
      • mmap with guard pages

An example of how to provide import functions?

There's an example for invoking a function from an instance, but with so little documentation, one has to dig through the code to figure out how to provide host functions, and even then, I'm not quite sure I understand how to go about it.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.