Coder Social home page Coder Social logo

quickjs-emscripten-sync's Introduction

quickjs-emscripten-sync

CI codecov

Build a secure plugin system in web browsers

This library wraps quickjs-emscripten and provides a way to sync object state between the browser and sandboxed QuickJS.

  • Exchange and sync values between the browser (host) and QuickJS seamlessly
    • Primitives (number, boolean, string, symbol)
    • Arrays
    • Functions
    • Classes and instances
    • Objects with prototypes and any property descriptors
    • Promises
  • Expose objects as a global object in QuickJS
  • Marshaling limitation for specific objects
  • Register a pair of objects that will be considered the same between the browser and QuickJS
npm install quickjs-emscripten quickjs-emscripten-sync
import { getQuickJS } from "quickjs-emscripten";
import { Arena } from "quickjs-emscripten-sync";

class Cls {
  field = 0;

  method() {
    return ++this.field;
  }
}

const ctx = (await getQuickJS()).newContext();
const arena = new Arena(ctx, { isMarshalable: true });

// We can pass objects to the context and run code safely
const exposed = {
  Cls,
  cls: new Cls(),
  syncedCls: arena.sync(new Cls()),
};
arena.expose(exposed);

arena.evalCode(`cls instanceof Cls`); // returns true
arena.evalCode(`cls.field`);          // returns 0
arena.evalCode(`cls.method()`);       // returns 1
arena.evalCode(`cls.field`);          // returns 1

arena.evalCode(`syncedCls.field`);    // returns 0
exposed.syncedCls.method();           // returns 1
arena.evalCode(`syncedCls.field`);    // returns 1

arena.dispose();
ctx.dispose();

Example code is available as the unit test code.

Operating Environment

  • Web browsers that support WebAssembly
  • Node.js

If you want to run quickjs-emscripten and quickjs-emscripten-sync in a web browser, they have to be bundled with a bundler tool such as webpack, because quickjs-emscripten is now written in CommonJS format and web browsers cannot load it directly.

Usage

import { getQuickJS } from "quickjs-emscripten";
import { Arena } from "quickjs-emscripten-sync";

(async function() {
  const ctx = (await getQuickJS()).newContext();

  // init Arena
  // ⚠️ Marshaling is opt-in for security reasons.
  // ⚠️ Be careful when activating marshalling.
  const arena = new Arena(ctx, { isMarshalable: true });

  // expose objects as global objects in QuickJS context
  arena.expose({
    console: {
      log: console.log
    }
  });
  arena.evalCode(`console.log("hello, world");`); // run console.log
  arena.evalCode(`1 + 1`); // 2

  // expose objects but also enable sync
  const data = arena.sync({ hoge: "foo" });
  arena.expose({ data });

  arena.evalCode(`data.hoge = "bar"`);
  // eval code and operations to exposed objects are automatically synced
  console.log(data.hoge); // "bar"
  data.hoge = "changed!";
  console.log(arena.evalCode(`data.hoge`)); // "changed!"

  // Don't forget calling arena.dispose() before disposing QuickJS context!
  arena.dispose();
  ctx.dispose();
})();

Marshaling Limitations

Objects are automatically converted when they cross between the host and the QuickJS context. The conversion of a host's object to a handle is called marshaling, and the conversion of a handle to a host's object is called unmarshaling.

And for marshalling, it is possible to control whether the conversion is performed or not.

For example, exposing the host's global object to QuickJS is very heavy and dangerous. This exposure can be limited and controlled with the isMarshalable option. If false is returned, just undefined is passed to QuickJS.

import { Arena, complexity } from "quickjs-emscripten-sync";

const arena = new Arena(ctx, {
  isMarshalable: (target: any) => {
    // prevent passing globalThis to QuickJS
    if (target === window) return false;
    // complexity is a helper function to detect whether the object is heavy
    if (complexity(target, 30) >= 30) return false;
    return true; // other objects are OK
  }
});

arena.evalCode(`a => a === undefined`)({});       // false
arena.evalCode(`a => a === undefined`)(window); // true
arena.evalCode(`a => a === undefined`)(document); // true

The complexity function is useful to detect whether the object is too heavy to be passed to QuickJS.

Security Warning

QuickJS has an environment isolated from the browser, so any code can be executed safely, but there are edge cases where some exposed objects by quickjs-emscripten-sync may break security.

quickjs-emscripten-sync cannot prevent such dangerous case, so PLEASE be very careful and deliberate about what you expose to QuickJS!

Case 1: Prototype pollution

import { set } from "lodash-es";

arena.expose({
  danger: (keys, value) => {
    // This function may cause prototype pollution in the browser by QuickJS
    set({}, keys, value)
  }
});

arena.evalCode(`danger("__proto__.a", () => { /* injected */ })`);

Case 2: Unintended HTTP request

It is very dangerous to expose or use directly or indirectly the window object, localStorage, fetch, XMLHttpRequest ...

This is because it enables the execution of unintended code such as XSS attacks, such as reading local storage, sending unintended HTTP requests, and manipulating DOM objects.

arena.expose({
  // This function may cause unintended HTTP requests
  danger: (url, body) => {
    fetch(url, {
      method: "POST",
      headers: {
        "Content-Type": "application/json"
      },
      body: JSON.stringify(body)
    });
  }
});

arena.evalCode(`danger("/api", { dangerous: true })`);

By default, quickjs-emscripten-sync doesn't prevent any marshaling, even in such cases. And there are many built-in objects in the host, so please note that it's hard to prevent all dangerous cases with the isMarshalable option alone.

API

Arena

The Arena class manages all generated handles at once by quickjs-emscripten and automatically converts objects between the host and the QuickJS context.

new Arena(ctx: QuickJSContext, options?: Options)

Constructs a new Arena instance. It requires a quickjs-emscripten context initialized with quickjs.newContext().

Options accepted:

type Options = {
  /** A callback that returns a boolean value that determines whether an object is marshalled or not. If false, no marshaling will be done and undefined will be passed to the QuickJS VM, otherwise marshaling will be done. By default, all objects will be marshalled. */
  isMarshalable?: boolean | "json" | ((target: any) => boolean | "json");
  /** Pre-registered pairs of objects that will be considered the same between the host and the QuickJS VM. This will be used automatically during the conversion. By default, it will be registered automatically with `defaultRegisteredObjects`.
   *
   * Instead of a string, you can also pass a QuickJSHandle directly. In that case, however, you have to dispose of them manually when destroying the VM.
   */
  registeredObjects?: Iterable<[any, QuickJSHandle | string]>;
  /** Register functions to convert an object to a QuickJS handle. */
  customMarshaller?: Iterable<
    (target: unknown, ctx: QuickJSContext) => QuickJSHandle | undefined
  >;
  /** Register functions to convert a QuickJS handle to an object. */
  customUnmarshaller?: Iterable<
    (target: QuickJSHandle, ctx: QuickJSContext) => any
  >;
  /** A callback that returns a boolean value that determines whether an object is wrappable by proxies. If returns false, note that the object cannot be synchronized between the host and the QuickJS even if arena.sync is used. */
  isWrappable?: (target: any) => boolean;
  /** A callback that returns a boolean value that determines whether an QuickJS handle is wrappable by proxies. If returns false, note that the handle cannot be synchronized between the host and the QuickJS even if arena.sync is used. */
  isHandleWrappable?: (handle: QuickJSHandle, ctx: QuickJSContext) => boolean;
  /** Compatibility with quickjs-emscripten prior to v0.15. Inject code for compatibility into context at Arena class initialization time. */
  compat?: boolean;
}

Notes:

isMarshalable: Determines how marshalling will be done when sending objects from the host to the context. Make sure to set the marshalling to be the minimum necessary as it may reduce the security of your application. Please read the section on security above.

  • "json" (default, safety): Target object will be serialized as JSON in host and then parsed in context. Functions and classes will be lost in the process.
  • false (safety): Target object will not be always marshalled as undefined.
  • (target: any) => boolean | "json" (recoomended): You can control marshalling mode for each objects. If you want to do marshalling, usually use this method. Allow partial marshalling by returning true only for some objects.
  • true (risky and not recommended): Target object will be always marshaled. This setting may reduce security.

registeredObjects: You can pre-register a pair of objects that will be considered the same between the host and the QuickJS context. This will be used automatically during the conversion. By default, it will be registered automatically with defaultRegisteredObjects. If you want to add a new pair to this, please do the following:

import { defaultRegisteredObjects } from "quickjs-emscripten-sync";

const arena = new Arena(ctx, {
  registeredObjects: [
    ...defaultRegisteredObjects,
    [Math, "Math"]
  ]
});

Instead of a string, you can also pass a QuickJSHandle directly. In that case, however, you have to dispose of them manually when destroying the context.

dispose()

Dispose of the arena and managed handles. This method won't dispose the context itself, so the context has to be disposed of manually.

evalCode<T = any>(code: string): T | undefined

Evaluate JS code in the context and get the result as an object on the host side. It also converts and re-throws error objects when an error is thrown during evaluation.

executePendingJobs(): number

Almost same as ctx.runtime.executePendingJobs(), but it converts and re-throws error objects when an error is thrown during evaluation.

expose(obj: { [k: string]: any })

Expose objects as global objects in the context.

By default, exposed objects are not synchronized between the host and the context. If you want to sync an objects, first wrap the object with sync method, and then expose the wrapped object.

sync<T>(target: T): T

Enables sync for the object between the host and the context and returns objects wrapped with proxies.

The return value is necessary in order to reflect changes to the object from the host to the context. Please note that setting a value in the field or deleting a field in the original object will not synchronize it.

register(target: any, code: string | QuickJSHandle)

Register a pair of objects that will be considered the same between the host and the QuickJS context.

unregisterAll(targets: Iterable<[any, string | QuickJSHandle]>)

Execute register methods for each pair.

unregister(target: any)

Unregister a pair of objects that were registered with registeredObjects option and register method.

unregisterAll(targets: Iterable<any>)

Execute unregister methods for each target.

defaultRegisteredObjects: [any, string][]

Default value of registeredObjects option of the Arena class constructor.

complexity(target: any, max?: number): number

Measure the complexity of an object as you traverse the field and prototype chain. If max is specified, when the complexity reaches max, the traversal is terminated and it returns the max. In this function, one object and function are counted as a complexity of 1, and primitives are not counted as a complexity.

Advanced

How to work

quickjs-emscripten can execute JS code safely, but it requires to deal with a lot of handles and lifetimes. Also, when destroying the context, any un-destroyed handle will result in an error.

quickjs-emscripten-sync will automatically manage all handles once generated by QuickJS context in an Arena class. And it will automatically "marshal" objects as handles and "unmarshal" handles as objects to enable seamless data exchange between the browser and QuickJS. It recursively traverses the object properties and prototype chain to transform objects. A function is called after its arguments and this arg are automatically converted for the environment in which the function is defined. The return value will be automatically converted to match the environment of the caller. Most objects are wrapped by proxies during conversion, allowing "set" and "delete" operations on objects to be synchronized between the browser and QuickJS.

Limitations

Class constructor

When initializing a new instance, it is not possible to fully proxy this arg (a.k.a. new.target) inside the class constructor. Therefore, after the constructor call, the fields set for this are re-set to this on the context side. Therefore, there may be some edge cases where the constructor may not work properly.

class Cls {
  constructor() {
    this.hoge = "foo";
  }
}

arena.expose({ Cls });
arena.evalCode(`new Cls()`); // Cls { hoge: "foo" }

Operation synchronization

For now, only the set and deleteProperty operations on objects are subject to synchronization. The result of Object.defineProperty on a proxied object will not be synchronized to the other side.

License

MIT License

quickjs-emscripten-sync's People

Contributors

kawaite avatar keiya01 avatar namuol avatar rot1024 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

quickjs-emscripten-sync's Issues

Doesn't work with asyncify

Would it be possible for this library to support asyncify? Currently if I want to marshal an asyncified function I can't do that e.g.

  vm.arena.expose({
    http: {
      fetchSync: vm.newAsyncifiedFunction(async () => {
        return await fetch(url).then(r => r.text())
      }),
    },
  })

[C to host error: returning null] Error: QuickJSContext had no callback with id -xxxx

When repeating calling a function, an error occurs from quickjs-emscripten.

import { getQuickJS } from "quickjs-emscripten";
import { expect, test } from "vitest";
import { Arena } from ".";

test(
  "repeated function",
  async () => {
    const rt = (await getQuickJS()).newRuntime();
    const ctx = rt.newContext();
    const arena = new Arena(ctx, { isMarshalable: true });

    vm.expose({ hoge: () => {} });
    // error happens from 3926 times on my machine
    for (let i = 0; i < 3926; i++) {
      // arg should be an object
      arena.evalCode(`hoge({})`);
    }

    arena.dispose();
    ctx.dispose();
    rt.dispose();
  },
  1000 * 60,
);

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Warning

These dependencies are deprecated:

Datasource Name Replacement PR?
npm @vitest/coverage-c8 Unavailable

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

github-actions
.github/workflows/ci.yml
  • actions/checkout v3
  • actions/setup-node v3
  • codecov/codecov-action v2
  • actions/upload-artifact v3
.github/workflows/pr_title.yml
  • amannn/action-semantic-pull-request v4
npm
package.json
  • @vitest/coverage-c8 ^0.28.5
  • eslint ^8.22.0
  • eslint-config-reearth ^0.2.1
  • prettier ^2.7.1
  • quickjs-emscripten ^0.21.0
  • typescript ^4.8.2
  • vite ^4.1.2
  • vite-plugin-dts ^2.0.0-beta.1
  • vitest ^0.28.5
  • quickjs-emscripten *
  • node >=12

  • Check this box to trigger a request for Renovate to run again on this repository

Preventing return of `globalThis`

currently a script like 'return globalThis' will attempt to return the globalThis object, which is huge and slow. How can we prevent this being attempted ?

Garbage collection error when trying to dispose of date in array

  test.only("Date in array", async () => {
    const ctx = (await getQuickJS()).newContext();
    const arena = new Arena(ctx, { isMarshalable: true });
    const scope = { results: [new Date(2022, 7, 26)] };
    arena.expose(scope);

    const evalResult = arena.evalCode("results[0]");
    console.log(evalResult);
    expect(evalResult).instanceOf(Date);

    arena.dispose();
    ctx.dispose();
  });

When running this test, an error is thrown:

RuntimeError: Aborted(Assertion failed: list_empty(&rt->gc_obj_list), at: quickjs/quickjs.c,1983,JS_FreeRuntime). Build with -s ASSERTIONS=1 for more info.

My best guess is the quickjs handle isn't being properly disposed of, but i'm not 100% sure. Working on a PR, but thought I'd flag here incase anyone had ideas.

Possible to synchronize vm -> host -> vm?

Hi

I'm trying to compare objects which crosses from VM -> host -> VM. This seems to fail. See the example below. Is it possible to add support for this? :)

Kind regards, Emil

(async () => {
    const vm = (await getQuickJS()).createVm();
    const arena = new Arena(vm, { isMarshalable: true });

    function test(obj) {
        return obj;
    }

    arena.expose({ test });
    console.log(arena.evalCode(`let foo = {}; foo === test(foo)`)); // I want this to log 'true', but it logs 'false'

    arena.dispose();
    vm.dispose();
})();

Also, thanks for this great library! :)

Unset all values created in arena

Currently a call to arena dispose doesn't remove variables from the context.

const arena1 = new Arena(vm, {
    isMarshalable: 'json'
})
arena1.expose({foo: 'bar'})
arena1.evalCode(`return foo`) // returns bar as expect
arena1.dispose()


const arena2 = new Arena(vm, {
    isMarshalable: 'json'
})
// I wasn't existing this to work after the dispose above
arena2.evalCode(`return foo`) // returns 'bar' unexpected

Is it possible to reset the arena? I'd like to share context between executions, as we apply polyfills to the run time, this takes a long time so I was trying to share the context between a few different evaluations.

Memory leak when only using marshalling

Issue

When having code that returns data from VM -> Host, the memory used will keep on increasing each time the evalCode call is made.

Additional context

"quickjs-emscripten": "^0.23.0",
"quickjs-emscripten-sync": "^1.5.2",

We are using this package to marshal a lot of classes, objects, functions from the Host -> VM. Because of this, we are using a single long-lived QuickJsContext.

Our host needs to call methods on the instances of classes created inside the VM, these functions may return either simple data objects or Promise containing simple data and will use functions made available from the Host -> VM.

We have no use case for using the syncing functionality provided by this package.

It appears that because of the syncing capabilities of this package, memory is being retained by the QuickJsRuntime for each call made to the ´evalCode´ that needs to return some data.

What did I already try

Disable isHandleWrappable and isWrappable. While this appeared to work at first glance, it however doesn't work when the code is async/await. It also doesn't work when using moduleLoader.

Expected

No memory is being retained when not using the syncing capabilities.

Reproduce

  • memory no leak will log an increase of 948.
  • memory leak will log a number much higher.

If you change the amount of iterations of the arena.evalCode("globalThis.test.check()");, the memory used for the no leak scenario will remain at 948 while the leak scenario will keep on increasing for each iteration.

  test("memory no leak", async () => {
    const ctx = (await getQuickJS()).newContext();
    const arena = new Arena(ctx, { isMarshalable: true });

    const getMemory = () => {
      const handle = ctx.runtime.computeMemoryUsage();
      const mem = ctx.dump(handle);
      handle.dispose();
      return mem;
    };

    arena.evalCode(`globalThis.test = {
      check: () => {
        return {
          id: 'some id',
          data: 123
        };
      }
    }`);

    const memoryBefore = getMemory().memory_used_size as number;

    for (let i = 0; i < 10; i++) {
      const handle = ctx.unwrapResult(ctx.evalCode("globalThis.test.check()"));
     const data = ctx.dump(handle);
     handle.dispose();

     expect(data).toStrictEqual({id: 'some id', data: 123});
    }

    const memoryAfter = getMemory().memory_used_size as number;

    console.log("Allocation increased %d", memoryAfter - memoryBefore);

    arena.dispose();
    ctx.dispose();
  });

  test("memory leak", async () => {
    const ctx = (await getQuickJS()).newContext();
    const arena = new Arena(ctx, { isMarshalable: true });

    const getMemory = () => {
      const handle = ctx.runtime.computeMemoryUsage();
      const mem = ctx.dump(handle);
      handle.dispose();
      return mem;
    };

    arena.evalCode(`globalThis.test = {
      check: () => {
        return {
          id: 'some id',
          data: 123
        };
      }
    }`);

    const memoryBefore = getMemory().memory_used_size as number;

    for (let i = 0; i < 10; i++) {
      const data = arena.evalCode("globalThis.test.check()");
      expect(data).toStrictEqual({id: 'some id', data: 123});
    }

    const memoryAfter = getMemory().memory_used_size as number;

    console.log("Allocation increased %d", memoryAfter - memoryBefore);

    arena.dispose();
    ctx.dispose();
  });

Support for async/Promises

Hey there,

I've had a blast using your library but noticed promises don't really work (resolving them inside QuickJS). Is there a limitation on this, or is it possible to use them?

Thanks

Error for reference: TypeError: Promise.prototype.then called on incompatible Proxy

evalCode async code problem

const execute = async () => {
try {
const res = await someAsyncOperation();
console.log(res)
} catch (e) {
console.log(e)
}
};
execute();
i write this code to run (use arena.evalCode), but never return any res, if i should do something more? I have expose someAsyncOperation

Not possible to expose fetch?

Hello,

This module has some really useful features on top of quickjs-emscripten, thank you!

In particular, I'm trying to enable fetch. Your docs describe how it's dangerous to expose it into the arena. However, it appears the arena doesn't support promises, which would mean fetch can never work, if I'm understanding correctly.

Is that right? Why do the docs talk about exposing fetch when promises don't work? Or am I missing something?

Thank you,
Joseph

include Buffer in registerdObjects

Hello, I'm trying to run a script like this

return Buffer.from('abc').toString('base64')

I've tried the following approaches to this

 import {Buffer} from 'node:buffer'
 
  const vm = QuickJS.newContext()
  const arena = new Arena(vm, {
    isMarshalable: true,
    registeredObjects: [
      ...defaultRegisteredObjects,
      [Buffer, 'Buffer'],
      [Buffer.prototype, 'Buffer.prototype']
    ]
  })
  value = arena.evalCode(`return Buffer.from('abc').toString('base64')`)
  // throws ReferenceError: 'Buffer' is not defined

and

import {Buffer} from 'node:buffer'

const vm = QuickJS.newContext()
arena.expose({
  Buffer: Buffer,
  ...payload
})
value = arena.evalCode(`return Buffer.from('abc').toString('base64')`)
// throws Error running script: return Buffer.from('abc').toString('base64'): argument must be a buffer

Am I on the right track here or doing something silly ?

Thanks 🙏

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.