`SUPPORT_BIG_ENDIAN=1` causes internal compiler error `ReferenceError: maybeExport is not defined` about emscripten

[merceyz]

Compile on install can be done but then you risk the host system not having the required tools or correct version of tools to build the library.

Some npm packages only ship pre-compiled versions and fail if it doesn't have one for the host and Node.js version, while some fall back to compile on install.

It's easier for us to ignore all that and compile once to WASM.
It's also faster for our users since they don't have to compile anything when installing Yarn.

Also, just to confirm, the SUPPORT_BIG_ENDIAN setting means that result will run on both big and little endian with a single build?

Yes.

Hopefully it doesn't come with too much of a performance hit.

When support for big-endian was added to Yarn the benchmarks indicated almost no impact on performance yarnpkg/berry#3669 (comment).

[sbc100]

It looks like that function was renamed in #21439 but that call was overlooked. I guess we don't have any testing for that setting :-/

Can I ask what your target is that you are using that option?

[merceyz]

Can I ask what your target is that you are using that option?

Of course!

Yarn compiles libzip to WASM for Node.js and needs to support big-endian systems.
I don't know specifically what systems people run it on but I know at least big-endian s390x systems are in use.

We don't have access to a big-endian system so our CI emulates one while Node.js CITGM validates that it works on a real machine.

[sbc100]

Thanks! Thats good to know. I didn't know yarn was using wasm like this. BTW, can I ask why you/they didn't just go with a native node binary? What is the advantage they/you seen in using wasm under node? (I'm not saying there are not advantages, I'm just wondering why you made the choice).

[merceyz]

The main reason is reproducibility. We need the output of libzip to be identical on all systems since we checksum the zip files to see if they've been tampered with or corrupted.

It's also easier to compile to WASM once and use it everywhere Node.js works than to make a native Node.js addon for each and every architecture, platform, and libc implementation that Node.js runs on.

[sbc100]

I see, I'm not very familiar with node addon.

Does that mean that npm doesn't generally do compile on install for things like? Does it instead rely on your to compile native extension for evey possible arch up front? That does sounds hard/impossible.

Also, just to confirm, the SUPPORT_BIG_ENDIAN setting means that result will run on both big and little endian with a single build?

I guess we should add more testing of this setting if folks like you are depending on it so much. Hopefully it doesn't come with too much of a performance hit.

[sbc100]

Thanks for the useful data points!