Generate compile_commands.json about b2

[grisumbras]

LGTM
Also, AFAIK, link commands aren't needed.

[vinniefalco]

What is this "load an existing compile_commands.json?" Why?

[grafikrobot]

What is this "load an existing compile_commands.json?" Why?

Because B2 can do very fine grained incremental builds. And if you want to get a full picture of the project you can't throw away the history of what you previously compiled on each b2 invocation. Especially if it's an invocation that's a no-op, which would create an empty json file.

[vinniefalco]

That is a nice to have feature but maybe not strictly required for an initial proof of concept.

[grisumbras]

I think, the described approach is actually not needed. Running with different build requests, and appending commands to compile_commands.json is both unnecessary complexity and potentially problematic, when e.g. some targets are removed from the build scripts.

Rather, I think something like b2 variant=debug,release --compile-db should create a new file for all jam targets it considers for update (not necessarily updates). Maybe we also don't want it to actually try updating them, although there is already -n flag for that.

There's also a question what to do with ancillary targets e.g. created by ac module. Maybe potentially add an engile rule that marks a target as hidden?

Now, that I think of it, what if we can mark Jam targets as watched (maybe by setting a variable on them), and then their actions get recorded into some list in the form of command output sources?

[grafikrobot]

After my many questions in slack, adjusted plan:

1. Add a feature to register action pre-execution hooks in the engine side. Such hooks would be given the action script string and can process the string. Possibly returning a different string to execute.
2. Add a action pre-exec hook to watch for per-toolset compile (and link?) commands that extracts the single command line invocation (by matching a regex in the action script). Control would be by adding a target variable COMMAND_DATABASE that sets a regex that extracts the command.
3. Change, initially, the gcc/clang toolset to set the variable for each appropriate action.
4. Add a command line arg --command-database=json that simulates the build request (i.e. equivalent to -n -a). That collects all the registered action commands and writes out the resulting json to <build-dir>/compile_commands.json.
5. Add a command line arg --command-database-out=<filename> to specify an override to the file name and location.

[grafikrobot]

Step 0? Structured data output to JSON complete: c30bce8

[grafikrobot]

WIP here.. https://github.com/bfgroup/b2/tree/feature/compiledb