jerch / inwasm Goto Github PK

Currently recompilation will be skipped, if the definition did not change and -f (force) was not set. There are several issues attached to this cache logic:

• does only work for inlined code, any code changes outside will not be spotted (eg. custom runner example will not recompile atm)
• some parts of the definition object are not yet correctly tracked, thus changes dont update compilation

The first point is a tricky one - we basically end up in make realms and would have to track a tree of outside source files, hmm.

There is more attached to placing source files outside - if a code change happens outside without any change in the TS file, inwasm will silently skip that wasm build due to being already injected. Furthermore we already lost the definition association due to the inline writing.

Ideas to overcome both issues:

• make like file tracking:
  • Allow folder globbing in definition and track files' mtime? Somewhat cumbersome to get done right ...
  • noCache setting at definition - This way ppl can just use make in a custom runner with its fancy tracking mechs.
• lost definition due to inline writing:
  • preserve definition commented out, so a follow-up run can still inspect/restore it
  • needs special care for the second module import to not end up in a compile loop (place some sort of inwasm run token there?)

• endless loop with noCache=true - solvable by content hashing
• not tracking trackedChanges entries yet - needs second watcher to trigger associated rebuilt
• chokidar watch does nothing, if folder of pattern does not exist yet

The path configurations need some tweaking:

• make builds path configurable - Currently it is assumed, that inwasm is called from project root and builds get simply written to ./inwasm-builds. Thats not good:
  • breaks, if inwasm was not called from project root --> needs to find out the real project root (builds in project root should stay the default)
  • out of project folder builds not possible --> needs an explicit conf overwrite
• SDK path for autoinstall needs to be configurable

The problems above arise from more complicated projects with subprojects like in xtermjs - currently inwasm is only used there in addons, which form their own projects with a package.json. Ideally we can just install inwasm on the main project and delegate the builds output to the sub projects. The SDKs on the other hand should only be loaded once globally. This raises the question, whether we should implement a "parent-project" mode (descending logic to sub projects) or keep inwasm sub-project-local (ascending logic to parent project), or both.

Also see xtermjs/xterm.js#4646.

left to do before going beta:

• fix wabt paths for inwasm from package
• create emscripten and clang c++ runners
• ESM support for inwasm
• test on macos (gh action?)
• windows support
• fix recompilation cache logic, see #10
• more tests

Repro:

• create trackChanges with a certain source file
• run inwasm -w
• provoke compile error by entering nonsense code in that file
• fix error or edit something else in file --> watcher does not retry compilation

The current tracking is enough to stop the sdks from being loaded in CI under linux and macos, but not under windows. The reason is prolly the different line terminator, which leads to other hash values than the tracked ones.

Normalizing between CRNLs and NLs is really weird, so how about a cmdline flag to indicate in isolated CI runs, that the existing prebuilt binary should be used, disregarding all other hash based change tracking? This makes the skip logic pretty straight forward, and can also be used later on by any repo checkout, that never deals with wasm source code at all.

Currently the watch mode operates only on initially existing files.

Reshape trackChanges into trackModified and add another setting trackContent for real file content hashing.

The latter is needed to avoid CI sdk pulling from a fresh repo checkout containing build targets (as needed in xterm.js serialize addon).