Local development setup, adjusting Closure compiler compilation level about what-if-tool

[stephanwlee]

Sorry that you had to go through painful debugging sessions to arrive to that conclusion. Few clarifications:

changes to the filesystem aren't detected and you have to kill the process...

1. For this, you can use ibazel. https://github.com/bazelbuild/bazel-watcher

bazel run whatev --be-faster

2. Bazel has a notion of compilation mode. https://docs.bazel.build/versions/master/command-line-reference.html#flag--compilation_mode It is up for our build macros/tools to honor that but we currently do not understand it.

3. It is not just the Closure Compiler that takes time; our Vulcanization walks through dependency graph (specified in <link rel="import">) and flattens the document. Of course, it is probably tiny compare to time we spend on JSComp.

4. (stating obvious) Bazel is awesome at caching artifacts based on the input. However, the Vulcanization step does not create any intermediate step and every line of JavaScript code change invokes TypeScript+JavaScript to be recompiled as whole. If there were intermediate steps (e.g., ts -> js transpilation), Bazel can do a better job at speeding the incremental builds.

5. TensorBoard, especially given that our tests were lacking, tried to make dev setup equal to prod setup so we do not make changes that happen to work during dev but not in prod (and we do not expect developers to manually validate the change works in prod).

[kevinrobinson]

@stephanwlee No worries. This is super helpful, thanks! 👍

So the root issue is Vulcanize.java can't do incremental builds. But between #2 and #5, it sounds like a Bazel-y way to support faster development builds would be to modify tensorboard_html_binary so that it respected the fastbuild compilation mode when passed as a flag (and avoided advanced mode compilation for one). Folks could use that in local development, but since it adds drift between dev/prod you haven't done that to this point, and understandably might not want to :)

Relatedly, I'm curious how you all develop now - do you have much faster build times on newer laptops, or are you offloading these build tasks to a big speedy Bazel build server in the 🌩️ ?

[stephanwlee]

Bazel can do distributed building (https://docs.bazel.build/versions/2.0.0/remote-execution.html) but I don't think I use that. My desktop machine tends to have more core and added parallelism definitely helps. My build at the end is closer to 20s but I wish it was closer to <1s or at most 10s.

There are other bundler strategies that we want to employ but those are more appropriate when we can actually trust the tests.

[jameswex]

Thanks both of you for this great discussion on compilation, as our WIT compilation story is obviously complex, due to our reliance on TB build rules which were required to support the complex build story for both TB and WIT/Facets (TS, JS, protos, goog.require, dep graphs, ...).

@kevinrobinson for now I think at the very least it makes sense to document the ability to use compilation level BUNDLE during development in our DEVELOPMENT.md. Do you wish to make that PR?

[kevinrobinson]

@jameswex Sure, #90 does that and I'll close this. Feel free to re-open if you want to - faster builds are always better, but I'm not planning at looking at that more right now.