It could help to always use JS for memsets during startup / that are always large about emscripten

[sbc100]

First of all I will just note that for mmap of file we already end up using the mmapAlloc JS function which calls zeroMemory which in turn calls HEAP8.fill, so I guess we are only talking about MAP_ANONYMOUS mappings here?

Its true that bulkmemory should be enabled by default soon, but you can also enable it today. Is it possible to enable it for all your users, or do you not have control over the flags they use?

Having said that I don't see why we wouldn't have a emscripten_memset_js just like we have a emscripten_memcpy_js. Would you be able to send a PR at add that?

Finally, if you are setting as lot calls to mmap(MAP_ANONYMOUS), it might be worth replacing them with a simple malloc instead since in emscripten MAP_ANONYMOUS is simply fake, and is strictly worse that just calling malloc + memset. Does you codebase have a fallback for when MAP_ANONYMOUS is not available?

[kg]

First of all I will just note that for mmap of file we already end up using the mmapAlloc JS function which calls zeroMemory which in turn calls HEAP8.fill, so I guess we are only talking about MAP_ANONYMOUS mappings here?

Sounds like it's just MAP_ANONYMOUS, yeah.

Its true that bulkmemory should be enabled by default soon, but you can also enable it today. Is it possible to enable it for all your users, or do you not have control over the flags they use?

I am hoping we'll be able to turn it on for everyone, but we've had issues turning on uncontroversial (to me) flags so far, so I don't know when we can make it the default. We still support disabling both WASM SIMD and WASM EH for users who have old Android or iOS devices, so bulk memory potentially adds a third configuration flag once we turn it on by default. When a developer flips one or both of those off, we re-link our whole runtime using emscripten on the developer's machine to apply the appropriate flags and link in the appropriate bits.

Having said that I don't see why we wouldn't have a emscripten_memset_js just like we have a emscripten_memcpy_js. Would you be able to send a PR at add that?

Yeah, I can put a PR together if it's wanted. I'll try to remember to get to that this week.

Finally, if you are setting as lot calls to mmap(MAP_ANONYMOUS), it might be worth replacing them with a simple malloc instead since in emscripten MAP_ANONYMOUS is simply fake, and is strictly worse that just calling malloc + memset. Does you codebase have a fallback for when MAP_ANONYMOUS is not available?

Thanks for the suggestion, it hadn't occurred to me that malloc could be meaningfully better here. Thankfully we don't have bare mmap calls in our codebase - they're wrapped, and the wrapper is wasm-specific - so it's possible I can turn all our mmap/munmap calls into malloc and free. I expect it will pay off, since my profiles don't show memset as a hotspot under malloc calls (malloc itself is something of a hotspot, but not as bad as memset.)

[sbc100]

Thanks for the suggestion, it hadn't occurred to me that malloc could be meaningfully better here. Thankfully we don't have bare mmap calls in our codebase - they're wrapped, and the wrapper is wasm-specific - so it's possible I can turn all our mmap/munmap calls into malloc and free. I expect it will pay off, since my profiles don't show memset as a hotspot under malloc calls (malloc itself is something of a hotspot, but not as bad as memset.)

So in your case the caller of mmap was not relying on the resulting pages actually being zero, and when you replace with malloc you don't need to also add memset?

[kg]

Thanks for the suggestion, it hadn't occurred to me that malloc could be meaningfully better here. Thankfully we don't have bare mmap calls in our codebase - they're wrapped, and the wrapper is wasm-specific - so it's possible I can turn all our mmap/munmap calls into malloc and free. I expect it will pay off, since my profiles don't show memset as a hotspot under malloc calls (malloc itself is something of a hotspot, but not as bad as memset.)

So in your case the caller of mmap was not relying on the resulting pages actually being zero, and when you replace with malloc you don't need to also add memset?

In these cases, the caller is relying on them being zero, but they're new "pages" that were allocated by sbrk/pre-allocated with initial heap size, and are already zero.

[sbc100]

Thanks for the suggestion, it hadn't occurred to me that malloc could be meaningfully better here. Thankfully we don't have bare mmap calls in our codebase - they're wrapped, and the wrapper is wasm-specific - so it's possible I can turn all our mmap/munmap calls into malloc and free. I expect it will pay off, since my profiles don't show memset as a hotspot under malloc calls (malloc itself is something of a hotspot, but not as bad as memset.)

So in your case the caller of mmap was not relying on the resulting pages actually being zero, and when you replace with malloc you don't need to also add memset?

In these cases, the caller is relying on them being zero, but they're new "pages" that were allocated by sbrk/pre-allocated with initial heap size, and are already zero.

In that case it sounds pretty risky to skip the memset after the malloc, since malloc makes no guarantees.

I wonder if we could improve our fake mmap(MAP_ANONYMOUS) such that it could know through some internal mechanism that it can sometimes skip the memset? Maybe unnecessary once we have emscripten_memset_js.

[sbc100]

BTW I am adding an underscore prefix to the internal memset/memcpy helpers: #21622