A more cursed technique... =) about array-buffer-detach

[bengl]

This is some interesting work. Thanks for digging in!

This proves to be ~30 times faster on my setup

Faster than not freeing them, and using regular ArrayBuffers, right?

I had explored putting this sort of approach together, with a new subclass of ArrayBuffer called "DisposableArrayBuffer", which would be allocated much like in your approach, but ultimately decided against it since the ability to do this without a native addon, or modifying Node.js itself, and for any arbitrary ArrayBuffer is very compelling. It also means that if you don't detach, the GC can still do its job later on as normal.

your detach technique does not seem to work for me in freeing up the memory for the wrapping ArrayBuffer in the hot loop so I see memory constantly growing

Are you using a custom ArrayBufferAllocator? Or the default V8 one? Or something akin to what Node.js does? I wonder if that's what makes the difference here.

I had to set the --allow-natives-syntax flag on the command line as v8 i am on barfs when i try to change the flags after initialising v8 platform.

If you're doing that, then you don't need to use the Function constructor. You can just put the natives syntax in your code directly, even from within your benchmarks. No need to wrap it at all.

[billywhizz]

i also tried this detach technique with the process pinned to a single core and the rate is pretty much same as the normal way of doing it - if even a tiny bit slower. so it's trading increased cpu usage (for GC, on another thread) up front against reduced memory usage as far as i can see.

[billywhizz]

i've been fiddling around with this approach and it's broken in various ways. trying to find an efficient (and safe) way to wrap external memory in v8.

fyi - i think the speed improvement is likely down to fact i am never writing to the memory and calloc always seems to return the same block of memory if i free it directly after and run in a tight loop.

[billywhizz]

btw - it turns out the memory leak i experienced was down to a current bug in v8 when pointer compression is enabled. thanks to the deno folks for documenting it!

[billywhizz]

another update. i built a new v8 static library on latest v8 beta branch which fixed the issue above, but meant i had to turn off pointer compression. i have verified the technique of yours is indeed faster in a tight loop than leaving v8 to deal with de-allocation, but the results from doing a separate call to calloc and then wrapping the memory in a buffer with no dispose callback are pretty insane. 🤯 over 30x faster.

i'll have to have a dig into v8 source to try to understand why. we are not touching the memory we are allocating so it may just be the fact that memory does not have to be filled with zeros each time around.

[billywhizz]

this is what the js benchmark looks like.

import { Bench } from 'lib/bench.js'
import { system } from 'lib/system.js'

const { wrapMemory, unwrapMemory, assert } = spin

const bench = new Bench()

let runs = 0
const size = 100 * 1024 * 1024

while (1) {
  runs = 6000

  for (let i = 0; i < 5; i++) {
    bench.start(`new ArrayBuffer ${size}`)
    for (let j = 0; j < runs; j++) {
      const buf = new ArrayBuffer(size)
      assert(buf.byteLength === size)
    }
    bench.end(runs)
  }

  runs = 6000

  for (let i = 0; i < 5; i++) {
    bench.start(`new ArrayBuffer w/unwrap ${size}`)
    for (let j = 0; j < runs; j++) {
      const buf = new ArrayBuffer(size)
      assert(buf.byteLength === size)
      unwrapMemory(buf)
      assert(buf.byteLength === 0)
    }
    bench.end(runs)
  }

  runs = 180000

  for (let i = 0; i < 5; i++) {
    bench.start(`calloc/wrap external ${size}`)
    for (let j = 0; j < runs; j++) {
      const address = system.calloc(1, size)
      const buf = wrapMemory(address, size, 0)
      assert(buf.byteLength === size)
      system.free(address)
    }
    bench.end(runs)
  }

  runs = 180000

  for (let i = 0; i < 5; i++) {
    bench.start(`calloc/wrap external w/unwrap ${size}`)
    for (let j = 0; j < runs; j++) {
      const address = system.calloc(1, size)
      const buf = wrapMemory(address, size, 0)
      assert(buf.byteLength === size)
      system.free(address)
      unwrapMemory(buf)
      assert(buf.byteLength === 0)
    }
    bench.end(runs)
  }

  runs = 6000

  for (let i = 0; i < 5; i++) {
    bench.start(`calloc/wrap internal ${size}`)
    for (let j = 0; j < runs; j++) {
      const address = system.calloc(1, size)
      const buf = wrapMemory(address, size, 1)
      assert(buf.byteLength === size)
    }
    bench.end(runs)
  }

  runs = 6000

  for (let i = 0; i < 5; i++) {
    bench.start(`calloc/wrap internal w/unwrap ${size}`)
    for (let j = 0; j < runs; j++) {
      const address = system.calloc(1, size)
      const buf = wrapMemory(address, size, 1)
      assert(buf.byteLength === size)
      unwrapMemory(buf)
      assert(buf.byteLength === 0)
    }
    bench.end(runs)
  }

  runs = 6000000

  for (let i = 0; i < 5; i++) {
    const address = system.calloc(1, size)
    bench.start(`wrap existing external ${size}`)
    for (let j = 0; j < runs; j++) {
      const buf = wrapMemory(address, size, 0)
      assert(buf.byteLength === size)
    }
    bench.end(runs)
    system.free(address)
  }

  runs = 6000000

  for (let i = 0; i < 5; i++) {
    const address = system.calloc(1, size)
    bench.start(`wrap existing external w/unwrap ${size}`)
    for (let j = 0; j < runs; j++) {
      const buf = wrapMemory(address, size, 0)
      assert(buf.byteLength === size)
      unwrapMemory(buf)
      assert(buf.byteLength === 0)
    }
    bench.end(runs)
    system.free(address)
  }
}

and the wrapMemory and unwrapMemory from C++

void spin::WrapMemory(const FunctionCallbackInfo<Value> &args) {
  Isolate* isolate = args.GetIsolate();
  uint64_t start64 = (uint64_t)Local<Integer>::Cast(args[0])->Value();
  uint32_t size = (uint32_t)Local<Integer>::Cast(args[1])->Value();
  void* start = reinterpret_cast<void*>(start64);
  int32_t free_memory = 0;
  if (args.Length() > 2) {
    free_memory = (int32_t)Local<Integer>::Cast(args[2])->Value();
  }
  if (free_memory == 0) {
    std::unique_ptr<BackingStore> backing = ArrayBuffer::NewBackingStore(
        start, size, v8::BackingStore::EmptyDeleter, nullptr);
    Local<ArrayBuffer> ab = ArrayBuffer::New(isolate, std::move(backing));
    args.GetReturnValue().Set(ab);
    return;
  }
  std::unique_ptr<BackingStore> backing = ArrayBuffer::NewBackingStore(
      start, size, spin::FreeMemory, nullptr);
  Local<ArrayBuffer> ab = ArrayBuffer::New(isolate, std::move(backing));
  args.GetReturnValue().Set(ab);
}

void spin::UnWrapMemory(const FunctionCallbackInfo<Value> &args) {
  Local<ArrayBuffer> ab = args[0].As<ArrayBuffer>();
  ab->Detach();
}