t-bre / ccdgen

Python script for generating compilation databases by capturing standard output from Make

Compile Commands Database GENerator

About

A Python script to generate a compile_commands.json database by capturing the output of make. This script was originally created to provide compilation databases for make based C/C++ projects in Visual Studio Code.

Installation

Latest PyPi release

pip install ccdgen

Usage

python3 -m ccdgen --extensions <arguments...> -- <your build command>

Arguments:

Option	Default	Description
-h, --help		Show help message and exit
-c, --compiler	(auto detect)	Specify compiler
-d, --dir	./	Working directory to run make from
-e, --extensions		Extension(s) for source files
-o, --output	./compile_commands.json	Output file

For example, to run make all as the build command for a C project:

python3 -m ccdgen --extensions .c -- make all

Example Visual Studio Code task:

{
    "version": "2.0.0",
    "tasks": [
        {
            "label": "ccdgen",
            "type": "shell",
            "command": "python",
            "osx": {
                "command": "python3"
            },
            "args": [
                "-m", "ccdgen",
                "--extensions", ".c",
                "--", "make", "all"
            ]
        }
    ]
}

Limitations

• The script relies on the Python standard library modules argparse, json, os, subprocess and sys.
• This script relies on make echoing the compiler commands it runs to stdout. If compiler commands are prefixed in the Makefile with @ or make is run in silent mode, the output cannot be captured.
• The build must succeed to generate a full compilation database, though warnings are not a problem.
• Currently only tested with Python 3.10 on macOS Ventura and Windows 10.

Other Tools

• CMake (since version 2.8.5) can be used as is to generate compile_commands.json by adding -DCMAKE_EXPORT_COMPILE_COMMANDS=ON when calling it. This only works for Unix Makefile builds.
• Bear is much more advanced tool for generating compilation databases for clang tooling. macOS, Linux and FreeBSD are currently supported.

ccdgen's Issues

feat: add ability to save commands in "arguments" format

Description

Add ability to export in "arguments" format instead of "command" format.

Additional Notes

See format recommendations from clang.

bug: paths with spaces are not escaped

Description

Paths with spaces are not escaped in the generated compile commands.

Steps To Reproduce

1. Run in folder with path including space.

Environment

os      : macOS
python  : 3.11

Expected Behaviour

Can run in directory with spaces in path name.

bug(build): `--dry-run` doesn't print build commands on Windows

Description

Running make -B --dry-run doesn't seem to output build commands to stdout on Windows.

Steps To Reproduce

1. Run on Windows with --dry-run.
2. Check stdout from make command.

Environment

os      : Windows
python  : 3.10

Expected Behaviour

Capture stdout in dry run mode on Windows.

Additional Notes

This is temporarily patched by #10 until a way of capturing stdout in dry run mode on Windows is found.

bug: errors are printed to stdout instead of stderr

Description

Errors (e.g. from make failing to run) are currently printed to stdout, when they should be printed to stderr.

Steps To Reproduce

1. Build with an invalid make command.
2. Error message goes to stdout.

Environment

os      : macOS
python  : 3.10

Expected Behaviour

Error messages printed to stderr.

feat: build wrapper mode

Description

• Currently, make is always run with --dry-run.
• It might be useful to have the option to run the script as a build wrapper.
• To allow for incremental builds, it will probably be necessary to load the existing database and update it.

bug(parser): compilers not detected when prefixed with path

Description

Compiler detection does not work when the program name is prefixed with a path (e.g. /usr/bin/clang++.

Steps To Reproduce

1. Set compiler to full path (e.g. /usr/bin/clang++).
2. Run without specifying compiler.
3. Compiler not detected.

Environment

os      : macOS
python  : 3.10

Expected Behaviour

Compiler detection works when compiler prefixed with path.

feat: auto detect source file extensions

Description

Automatically detect the extensions of source files, instead of requiring them to be specified from the CLI.

Additional Notes

• Should be possible to do this based on compiler arguments? At least for C/C++.
• Could also check the Makefile itself.