cppalliance / mrdocs

MrDocs: A Clang/LLVM tool for building reference documentation from C++ code and javadoc comments.

MrDocs

Install

This library depends on a recent version of LLVM. Here are the instructions to install LLVM with the settings required by this project.

git clone https://github.com/llvm/llvm-project.git
cd llvm-project
git checkout 5b959310b0fae723bd119ed8815bf1cb1a8c67d4
cmake -S llvm -B build -D LLVM_ENABLE_PROJECTS="clang;clang-tools-extra" -D CMAKE_BUILD_TYPE=Release -D LLVM_ENABLE_RTTI=ON -D CMAKE_INSTALL_PREFIX=/path/to/llvm+clang
cd build
cmake --build . -j <threads> --config Release
cmake --install .

Once LLVM is available in /path/to/llvm+clang, you can build and install mrdocs with:

git clone https://github.com/cppalliance/mrdocs
cd mrdocs
cmake -S . -B build -D CMAKE_BUILD_TYPE=Release -D LLVM_ROOT=/path/to/llvm+clang -D Clang_ROOT=/path/to/llvm+clang
cd build
cmake --build . -j <threads>
cmake --install .

Comparison

Doxygen:

• tries to work for many languages

• use the inferior libclang API

• old program with lots of technical debt

• not written by me

MrDocs:

• narrow and deep focus on C++ only

• uses clang’s unstable libtooling API:

  • Understands ALL C++: if clang can compile it, MrDocs knows about it

  • This includes up to C++20 and even experimental things in C++23

• brand-new program with no technical debt

• written by me

mrdocs's Issues

automatic metrics

It would be super nice to auto calculate the well-known sw metrics such as McCabe's Cyclomatic Complexity, Halstead's N, SLOCs, etc as part of generated docs.

Corpus is canonical

After the corpus is built it needs to be canonicalized, this means that all children need to be sorted against their siblings. The generator could specify whether or not it needs canonicalization (XML does).

define macros in config

It needs to be possible to define macros in the config file

additional Executor arguments from YML

There should be a YML key that lets the mrdox.yml file specify additional command line arguments to be appended to the passed command line arguments, which are then passed to the Executor of the front end action.

Self-Document MrDox

The MrDox project currently lacks documentation. It should be generated using MrDox itself

The high-level usage of the tool should be 2-step

The tool should be used similarly to C and C++ compilers: one invocation converts source to data, another invocation should assemble final result from data files created on step 1. This model is what build systems fundamentally support and hence no tfollowing it results in worse experience.

Consider a project with many files. If aforementioned 2-step model is not used, when a single file is changed than every source file has to be processed again. Even if the tool implements optimisation where it checks if a source file has been changed after the target was created, first of all it has to get a piece of information that was already available to the build system. But secondly, a C++ source file does not has to be itself for its contents to effectively change. Any of the headers it includes can effect such a change. So, the tool would have to recursively check all headers the file includes if they have been changec too. This work is already being done by build systems, it would be wasteful to implement it again and repeat it at runtime. And finally, a build system may have a different idea what "file has changed" means, for example Scons checks a file's hash rather then checking its last edit timestamp. Presumably, Scons users prefer or at least expect that semantics for "file has changed" rather than the one implemented inside mrdox.

In addition, there's an additional benefit to 2-step approach by itself. If the data format (currently XML) is documented, then other tools could be used to process it.

Allow arbitrary json compilation command files

The clang tooling only seems to allow the exact filename "compilation_database.json". The tool should support any filename ending with ".json", for example:

mrdox --public --doxygen --format=adoc --executor=all-TUs test.json

ToolResults is keyed by hex of USR?

This is a weird choice why use hex, instead of just keying on the USR directly?

mrdox should have a version that can be printed to the shell

[klemens@theodora cmake-build-debug]$ ./mrdox --version
LLVM (http://llvm.org/):
  LLVM version 15.0.7
  Optimized build.
  Default target: x86_64-redhat-linux-gnu
  Host CPU: skylake

read .git/config

Possibly way to get the repo Url. E.g.:

[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[remote "origin"]
	url = [email protected]:klemens-morgenstern/async.git
	fetch = +refs/heads/*:refs/remotes/origin/*
[branch "pages"]
	remote = origin
	merge = refs/heads/pages

Use native line endings in tests

It would be nice to use native line endings so that the .gitattributes file is not needed. Test text files should be stored with POSIX-style line endings in the git repository, and converted to native in the working directory.

typedefs in asciidoc

Emitting typedefs is unimplemented in the Asciidoc generator

Error reporting with SourceLocation

Existing usage of SourceLocation to report errors is broken.

Concurrency as a command line option

We may wish to allow a command line option for setting the maximum concurrency.

BitstreamReader::readVersion was never defined

It might be worth looking into whether we are writing the version correctly (or if it is even needed).

debug heap leak checking on MSVC

I have the code but I'm struggling with defining the architecture so I can include windows.h

mrdox can't self-dox

I export the compile_commands.json for MrDox itself but I get clang errors when mrdox tries to generate the AST

Variables support

Variables are completely missing, there is no code for visitation of VarDecl and the corresponding metadata.

--verbose option and common MrDoxCommandLineArguments

It would be nice to control the verbosity setting from the command line, and to have shared command-line settings or objects that the tool and tests can share for common options.

issue warnings when parameters are undocumented

This could be a setting in the yaml file

figure out javadoc briefs and param/tparam sorting

We want the Javadoc to be immutable but we also want param and tparam to be sorted out for convenience. And we want to know which paragraph is the brief, show it first, but skip it when iterating the description.

Dependencies should be pulled in using a package manager

Please evaluate if a package manager such as Conan (preferably) or VCPKG can be used to pull in required dependencies. That would significantly simplify the build process.

some kind of diagnostics for test successes

The test program should at least write the number of successful files tested. And if there is a failure, the number of failures and so forth. This could possibly be a command line switch.

Correct handling of UTF8 inputs

The source code needs to be audited for UTF8 correctness and fixed as needed.

_Bool?

For some inexplicable reason, clang emits booleans as _Bool.

config: source file list plus wildcards

We need a config key that specifies which source files should be compiled, this is in addition to "source root". There is no requirement that the compiled source files are children of the source root. The source file key is a list of files and directories with wildcards possible (maybe look at doxygen for this).

No closing command name in error case

When "@code" is used and there is no matching "@encode", or the next command is not "@Endcode", the function getCloseName is not reporting the encountered close tag, which is needed for diagnostics.

Support outbound links to third party documentation

For example, clicking on std::string_view could take the user to the appropriate cppreference page.

Find code that generates VerbatimLineComment

Boost.URL javadocs don't causeVerbatimLineComment to be emitted, we need to figure out what does.

unnecessary copies of ClangDocContext

It looks like ClangDocContext is copied several times needlessly.

However, it could be the case that in at least one of those cases, the copy is necessary so each thread has its own version.

Why XML?

Why does the tool produce XML? The only benefit of XML I can imagine is post-processing it with XSLT.

consider Describe and mp11

With Describe for the many enumerations provided by clang, llvm, and MrDox a lot of manual switch/boilerplate could be replaced with concise mp11 invocations.

normalize source Location

Source locations need to be normalized to forward slashes for tests to work

GitHub Actions

It would be very nice to have GitHub Actions within this repository, so common GitHub documentation workflows can be simplified.

I'd suggest 3 separate actions:

• build documentation
• publish to GH pages
• publish to GH wiki

Since MrDox outputs asciidoc pushing to the hidden wiki repositories (ie. [email protected]:cppalliance/mrdox.wiki.git for this repository once enabled) should yield usable results.

Antora Collector Extension

We need a collector extension that runs mrdox on content sources when the library playbook runs

Javadoc can be disengaged

Javadoc should have a state, bool has_value to distinguish between empty comments and an absence of any Javadoc comment.

extract template aliases

It seems like the template aliases in Boost.Buffers are not showing up in the HTML output

Corpus is const

Generators should receive a const Corpus; modification must be disallowed.

set up dogfooding on the LLVM libraries

As a robustness test we can set up a target to generate compile command and run mrdox on the clang and llvm source trees.

fail-tests (and warnings)

Eventually mrdox will have to emit errors of some kind for example if a javadoc is malformed or if there are duplicate javadoc definitions. For this we will need fail-tests.

We might also want warnings, such as when parameters are not documented in the source inputs.

Member function attributes (static, const, etc)

Need to distinguish static member functions versus regular member functions, and we should pick up all of the attributes of the function such as constexpr, noexcept, override, and the like.

Are data members indexed and do they need USRs?

Currently, data members of records are not inherited from SymbolInfo. That means there is no source location information and no USR, so these items cannot be indexed, hyperlinked, or have source information displayed.

yaml config

mrdox.yml has to have:
list of namespaces to ignore (e.,g. "detail")
later, we can make this a regex
list of namespaces we care about
e.g. "boost::url"

we don't want to dump shit from "boost::" if we are building docs for "boost::url"
the yml also has to have a command line for getting the compile-commands.json
it is the responsibility of the owner of the repo to tell mrdox how the fuck to build the commands
we might also have a list of macros to define, with optional values
Boost.URL sets "BOOST_URL_DOCS" when it builds the docs

avoid copies of TypedefInfo and EnumInfo

and then remove the copy members from List

FunctionInfo: total ordering

FunctionInfo needs a total ordering with respect to other FunctionInfo in the same scope:

first, on case-insensitive unqualified name
second, on the formal parameter list
third on the template parameter list

For relevant scopes we might need to insert the access specifier somewhere in there.

Test tool option to regenerate all xml

The test tool needs a command line option to overwrite all the test xml files with up to date content.

Examples

Please provide some examples and a boilerplate template repository to jumpstart projects using MrDox

Add plugins/generators via DLL

It should be possible for MrDox to load generators at runtime via DLL "plugins", so that authors of new formats do not have to build LLVM in order to write and test them.

There's no rush for this.

command line options for testing program

The mrdox_tests program needs a CommandLineOptions struct or whatever LLVM calls it. Right now it just assumes argv is an array of directories but we would like to have "verbosity" options and also options to control whether or not adoc file previews are generated along side the test inputs (these are gitignored).

review command line options and defaults

For example the --output directory default is "docs" , is this best practices