fybrik / crdoc

Generate markdown documentation from Kubernetes Custom Resource Definition (CRD) YAML files

Fybrik

Fybrik is a cloud native platform to control the usage of data within an organization and thus enable business agility while securing enterprise data. Fybrik encapsulates a containerized workload and intermediates all data flows both into the workload and out of the workload, enabling enforcement of policies defining what can be done with the data.

For more information please visit our website.

Repositories

Fybrik is composed of the following repositiores

• fybrik/fybrik. This is the main code repository. It hosts the core components, install artifacts, and sample programs. It includes:

  • manager This directory contains the code for the main operator that is responsible for the control plane of Fybrik.

• fybrik/arrow-flight-module. This is the code repository for read/write data access module based on Arrow/Flight.

• fybrik/mover. This is the code respository for an implicit copy module based on Apache SparkSQL.

Issue management

We use GitHub issues to track all of our bugs and feature requests.

crdoc's Issues

version flag

Hi, first of all thanks for a great project.
A small thing I'm missing though is a version flag. It's always nice to be able to see what version I'm running of a tool with the classical --version & -v flag.

Add anchorize function to templates

Add anchorize as a go template function that produces github-compatible anchor links. Use it in the included template (also for groups which are currently not clickable)

Refs:
https://github.com/gohugoio/hugo/blob/d90e37e0c6e812f9913bf256c9c81aa05b7a08aa/markup/blackfriday/convert.go#L70
https://github.com/gohugoio/hugo/blob/d90e37e0c6e812f9913bf256c9c81aa05b7a08aa/markup/goldmark/autoid.go#L35

Embed default template

Requires go 1.16

Prepare for organization rename

Go modules should use fybrik.io

frontmatter template has broken links

The https://github.com/grafana-operator/grafana-operator project uses crdoc and the frontmatter template to generate documentation.

This project has found that crdoc generates broken links. This can be seen by going to:
https://grafana-operator.github.io/grafana-operator/docs/api/

and clicking on many of the links.

Something like:

<a href="grafanadashboardstatus">status</a>

should be generated as:

<a href="#grafanadashboardstatus">status</a>

This problem does not occur for the markdown template

Consider providing a Docker image

To run crdoc as part of CI pipelines, it would be helpful to have a Docker/OCI image provided with crdoc installed. While of course everybody could simply create such an image on their own, having an official image, which is kept up-to-date by the maintainers, is a more "reliable" solution.

Do the maintainers think providing such an image is possible, for example, via GitHub Container Registry?

Consider providing a non-distroless Docker image

With #108 automatic builds of a Docker image were introduced. The provided Docker image is a distroless one, which should be absolutely fine (and probably the best option) for running crdoc in most situations.

Unfortunately, GitLab CI/CD requires images that contain a shell. I guess similar requirements also exist in other CI/CD tools.

Hence I'm asking whether a second image could be provided which builds upon alpine or similar. I looked at the GoReleaser docs and it seems to be possible to define multiple image with different Dockerfiles.

Error when the field `AdditionalProperties: true` is set.

I am trying to generate documents for our crds with your great tool. All in all it works fine, but if our crd contains the AdditionalProperties: true field, an error occurs and no documents are generated.

A simple example crd looks like this:

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: example.examples.com
spec:
  group: examples.com
  names:
    kind: example
  versions:
  - name: v1
    served: true
    storage: true
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            properties:
               values:
                  description: test description
                  type: object
                  additionalProperties: true # Error field
                  default: {}
  scope: Namespaced

Trying to create documents with crdoc --resources=./ --output=./test.md results in the following error message:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1195edc]

goroutine 1 [running]:
github.com/mesh-for-data/crdoc/pkg/builder.getTypeName(0xc0004f2200, 0xc00023de00, 0x0)
	/home/wetzel/go/pkg/mod/github.com/mesh-for-data/[email protected]/pkg/builder/builder.go:223 +0xdc
github.com/mesh-for-data/crdoc/pkg/builder.(*ModelBuilder).addTypeModels(0xc000213d30, 0xc000497f40, 0xc0004f0090, 0xc0001cb9c8, 0x13, 0xc0004f2200, 0x0, 0x0, 0xc0004ee0e0, 0x10)
	/home/wetzel/go/pkg/mod/github.com/mesh-for-data/[email protected]/pkg/builder/builder.go:140 +0x45
github.com/mesh-for-data/crdoc/pkg/builder.(*ModelBuilder).addTypeModels(0xc000213d30, 0xc000497f40, 0xc0004f0090, 0xc0004ee150, 0xc, 0xc0004f2000, 0x0, 0x0, 0x0, 0x0)
	/home/wetzel/go/pkg/mod/github.com/mesh-for-data/[email protected]/pkg/builder/builder.go:156 +0x3ba
github.com/mesh-for-data/crdoc/pkg/builder.(*ModelBuilder).addTypeModels(0xc000213d30, 0xc000497f40, 0xc0004f0090, 0xc0004ee080, 0x7, 0xc00023d600, 0x1, 0x1, 0x1d0cea0, 0x0)
	/home/wetzel/go/pkg/mod/github.com/mesh-for-data/[email protected]/pkg/builder/builder.go:156 +0x3ba
github.com/mesh-for-data/crdoc/pkg/builder.(*ModelBuilder).Add(0xc000213d30, 0xc0004982c0, 0xc0004ac970, 0x1)
	/home/wetzel/go/pkg/mod/github.com/mesh-for-data/[email protected]/pkg/builder/builder.go:93 +0x2a6
main.RootCmd.func2(0xc000394c80, 0xc0003fa4e0, 0x0, 0x2, 0x0, 0x0)
	/home/wetzel/go/pkg/mod/github.com/mesh-for-data/[email protected]/main.go:69 +0x274
github.com/spf13/cobra.(*Command).execute(0xc000394c80, 0xc000194160, 0x2, 0x2, 0xc000394c80, 0xc000194160)
	/home/wetzel/go/pkg/mod/github.com/spf13/[email protected]/command.go:852 +0x472
github.com/spf13/cobra.(*Command).ExecuteC(0xc000394c80, 0xc000000180, 0xc000213f78, 0x408025)
	/home/wetzel/go/pkg/mod/github.com/spf13/[email protected]/command.go:960 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
	/home/wetzel/go/pkg/mod/github.com/spf13/[email protected]/command.go:897
main.main()
	/home/wetzel/go/pkg/mod/github.com/mesh-for-data/[email protected]/main.go:103 +0x2b

If I remove the AdditionalProperties: true field, it works fine again. Any idea what could be the problem?

Field default value

I'd like to be able to show the default value of a field. For example, the default value of spec.replicas is 1 below.

    schema:
      openAPIV3Schema:
        properties:
          spec:
            type: object
            properties:
              replicas:
                default: 1
                format: int32
                minimum: 0
                type: integer

Tag creation is broken

crdoc/.github/workflows/tag.yml

Lines 44 to 49 in db283e2

	github.git.createRef({
	owner: context.repo.owner,
	repo: context.repo.repo,
	ref: "refs/tags/${{ github.event.inputs.version }}",
	sha: context.sha
	})

• Should now use github.rest.git.createRef
• The sha should change to the output of the previous step

It might be better to just use the tag option of commit and add step instead (alsp requires setting a PAT).

Broken links to Kubernetes API

Some templates create a link to https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.20/#objectmeta-v1-meta which does no longer exist. An easy fix would be to change these to https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.27/#objectmeta-v1-meta.

All relevant occurrences can be found here: https://github.com/search?q=repo%3Afybrik%2Fcrdoc%20v1.20&type=code

Support adding type information from go code

For CRDs that are generated from go code (e.g., using kubebuilder/operator-sdk) type information can be extracted from the code. Other tools do that already, the benefit would only be for projects that contain both CRDs generated from go code and other CRDs.

Upgrade to go 1.17 (Required by Kubernetes 0.23)

Generate GitHub Flavored Markdown tables

First, hi and thanks for the great effort you made with providing this tool. It looks very promising to us and seems to be the only tool available that allows generating a markdown reference from CRD YAML files.

Would it be possible to generate "real" markdown tables such as the ones from GitHub Flavored Markdown instead of the HTML code which is currently generated?

Support adding type information from openapi/jsonschema

For CRDs generated by https://github.com/mesh-for-data/openapi2crd the source openapi/jsonschema schema may include type information that we can show (e.g., the name of the schema or its title)

Installation script is broken

# sh ./get-crdoc.sh -b ./bin -d
fybrik/crdoc info checking GitHub for latest tag
fybrik/crdoc debug http_download https://github.com/fybrik/crdoc/releases/latest
fybrik/crdoc info found version: 0.6.3 for v0.6.3/Linux/x86_64
fybrik/crdoc debug downloading files into /tmp/tmp.whAYLrNb8E
fybrik/crdoc debug http_download https://github.com/fybrik/crdoc/releases/download/v0.6.3/crdoc_0.6.3_Linux_x86_64.tar.gz
fybrik/crdoc debug http_download_curl received HTTP status 404