`wazero.NewCompilationCache` creates a `CompilationCache` instance that's not really caching anything about wazero

[evacchi]

ah I see, we stumbled upon this behavior while solving a different issue. In this case it is safer to Compile separately than Instantiateing, this way the flag closeWithModule won't be set implicitly

[evacchi]

When an in-memory cache is configured, compilation does not actually occur for real, even though we do invoke compileModule internally.

wazero/cache_test.go

Lines 137 to 142 in 350e81e

	// Creates new cache instance and pass it to the config.
	c := NewCompilationCache()
	config := NewRuntimeConfig().WithCompilationCache(c)
	_foo := NewRuntimeWithConfig(ctx, config)
	_bar := NewRuntimeWithConfig(ctx, config)

What really happens is that if there is a hit at getCompiledModuleFromMemory() then we return early with the pre-compiled module.

e.g. see

wazero/internal/engine/wazevo/engine_cache.go

Lines 46 to 50 in 1b2fd85

	func (e *engine) getCompiledModule(module *wasm.Module, listeners []experimental.FunctionListener, ensureTermination bool) (cm *compiledModule, ok bool, err error) {
	cm, ok = e.getCompiledModuleFromMemory(module)
	if ok {
	return
	}

you can verify this easily if you try to compile a large module (which would take a sensible time compiling twice)

[martinlakov]

@evacchi , @mathetake - As explained in the description - problem is observed when running the same module twice, using two runtime instances that share an in-memory cache. The module is then compiled every single time and no caching occurs.

This, to me, is a bug. Considering the already referenced comment - this is not the desired behavior too.

[ncruces]

Did you share a reproducer for this one?

[martinlakov]

@ncruces, @evacchi, @mathetake - after a bit of play with the debugger - I've come across the root cause for the problem. When using Instantiate or InstantiateWithConfig (implicit module compilation) - the flag closeWithModule on the compiled module is being raised. As a result of that - after module instantiation is complete - the compiled module is being closed (hence removed from the in-memory cache).

One can say - this is a problem with the documentation and yes - docs would definitely appreciate a bit of love here. However - considering that caching is at play - my honest suggestion is that the flag closeWithModule should only be respected if no cache instance is configured for the said runtime. This will unify behaviors with the FS based cache where regardless of the closeWithModule flag cached entries can be recovered (from disk) and re-compilation avoided.

[martinlakov]

@evacchi - this behavior is both inconsistent & poorly documented. My humble proposal still stands - closeWithModule should only happen when no cache is configured for the runtime.

[evacchi]

I do not disagree, but meanwhile my suggestion should help you address the shortcoming without having to wait for an update

[martinlakov]

@evacchi - thank you!