Compiler failing at optimization step with a big .ts file about assemblyscript

[sebpiq]

Sorry for coming back so late to you ! I have upgraded to latest AssemblyScript version and indeed it works now ;)

Thanks so much for the help !

[CountBleck]

Here is the complete error it seems:

Cannot enlarge memory, requested 2147504128 bytes, but the limit is 2147483648 bytes!
Cannot enlarge memory, requested 2147508224 bytes, but the limit is 2147483648 bytes!
Aborted(native code called abort())

▌ Whoops, the AssemblyScript compiler has crashed during optimize :-(
▌
▌ Here is the stack trace hinting at the problem, perhaps it's useful?
▌
▌ RuntimeError: Aborted(native code called abort())
▌     at abort (file:///Users/blumiere/Projects/assemblyscript/node_modules/binaryen/index.js:422:15)
▌     at _abort (file:///Users/blumiere/Projects/assemblyscript/node_modules/binaryen/index.js:3478:7)
▌     at binaryen_wasm.wasm.MixedArena::allocSpace(unsigned long, unsigned long) (wasm://wasm/binaryen_wasm.wasm-021889a6:wasm-function[813]:0x71d58)
▌     at invoke_iiii (file:///Users/blumiere/Projects/assemblyscript/node_modules/binaryen/index.js:5722:40)
▌     at binaryen_wasm.wasm.wasm::optimizeBlock(wasm::Block*, wasm::Module*, wasm::PassOptions&, wasm::BranchUtils::BranchSeekerCache&) (wasm://wasm/binaryen_wasm.wasm-021889a6:wasm-function[5986]:0x309bda)
▌     at binaryen_wasm.wasm.wasm::Walker<wasm::MergeBlocks, wasm::UnifiedExpressionVisitor<wasm::MergeBlocks, void>>::doVisitBlock(wasm::MergeBlocks*, wasm::Expression**) (wasm://wasm/binaryen_wasm.wasm-021889a6:wasm-function[5895]:0x30659f)
▌     at binaryen_wasm.wasm.wasm::Walker<wasm::MergeBlocks, wasm::UnifiedExpressionVisitor<wasm::MergeBlocks, void>>::walk(wasm::Expression*&) (wasm://wasm/binaryen_wasm.wasm-021889a6:wasm-function[5891]:0x305335)
▌     at binaryen_wasm.wasm.wasm::WalkerPass<wasm::PostWalker<wasm::MergeBlocks, wasm::UnifiedExpressionVisitor<wasm::MergeBlocks, void>>>::runOnFunction(wasm::Module*, wasm::Function*) (wasm://wasm/binaryen_wasm.wasm-021889a6:wasm-function[5893]:0x306521)
▌     at invoke_viii (file:///Users/blumiere/Projects/assemblyscript/node_modules/binaryen/index.js:5733:33)
▌     at binaryen_wasm.wasm.wasm::PassRunner::runPassOnFunction(wasm::Pass*, wasm::Function*) (wasm://wasm/binaryen_wasm.wasm-021889a6:wasm-function[2178]:0x1723f2)
▌
▌ If you see where the error is, feel free to send us a pull request. If not,
▌ please let us know: https://github.com/AssemblyScript/assemblyscript/issues
▌
▌ Thank you!

[CountBleck]

It seems to work if binaryen.js is built with a maximum of 4 GiB of memory (which is the hard maximum according to wasm-ld, likely because it's Wasm32).

Here's the gzipped bla.wasm by the way.

The time needed on my 15" mid-2014 MacBook Pro: node ../assemblyscript/bin/asc -O bla.ts -o bla.wasm 484.24s user 1.51s system 102% cpu 7:55.49 total

[sebpiq]

Thanks so much @CountBleck ! I have a couple of questions then :

• when you say "built with a maximum of 4 GiB of memory" you mean the amount of memory that's available to binaryen for optimization ?
• how would you go about detecting that error and reporting it in a more user friendly manner ?

[CountBleck]

Yes, I mean the amount of memory available to Binaryen.

To detect that error, perhaps one could check the size of HEAP8 and compare it to the maximum.

Also, it might be good to bump the maximum memory in binaryen.js to 4 GiB regardless.

[sebpiq]

Also, it might be good to bump the maximum memory in binaryen.js to 4 GiB regardless.

How do you do that ? I see only a few options are forwarded to binaryen from asc

To detect that error, perhaps one could check the size of HEAP8 and compare it to the maximum.

Same ... any pointers on how I can access these ?

[MaxGraey]

Binaryen should use add_link_flag("-sMAXIMUM_MEMORY=4gb") for that. Just need to add this flag here:
https://github.com/WebAssembly/binaryen/blob/main/CMakeLists.txt#L337

[CountBleck]

Yeah, that's basically the flag I used.

How do you do that ? I see only a few options are forwarded to binaryen from asc

That's done when binaryen.js is built, using the flag Max said.

any pointers on how I can access these ?

That would likely be done within asc and not in your code.

[MaxGraey]

Also, we could try to update binaryen.js to latest. Perhaps there are some fixes, improvements and less memory leakage or consumption.

[CountBleck]

Also, we could try to update binaryen.js to latest. Perhaps there are some fixes, improvements and less memory leakage or consumption.

I have zero experience with updating AS's binaryen.js version, which is why I haven't just stuck the flag in the action yet.

Also, I tested AS with the latest version of binaryen.js, and it doesn't work with the default 2 GiB.

[CountBleck]

@sebpiq Sadly, I'm not sure if there's a reliable way to detect OOM. I might look into it further, but the next version of AS (released in a day or two) should allow your code to be built.

[sebpiq]

Thanks a lot !
Sorry I haven't responded before. I'll try with the newest version and see what I get.
Thanks also for all the great work with assemblyscript !

[MaxGraey]

It may also be memory leak in binaryen: WebAssembly/binaryen#6239