containersolutions / helm-convert

Convert Helm charts into Kustomize compatible package

Helm convert plugin

Charts are curated application definitions for Helm, this plugin let you convert existing charts into Kustomize compatible package.

Install

Helm plugin

$ helm plugin install https://github.com/ContainerSolutions/helm-convert

Binary without Helm

If you don't have Helm installed, you can just download the binary from the release page.

Usage

See helm convert --help for usage. Example:

# convert the stable/mongodb chart into Kustomize compatible package
helm convert --destination mongodb --name mongodb stable/mongodb

# convert chart from a url
helm convert https://s3-eu-west-1.amazonaws.com/coreos-charts/stable/prometheus-operator

# convert the stable/mongodb chart with a given values.yaml file
helm convert -f values.yaml stable/mongodb

# convert the stable/mongodb chart and override values using --set flag:
helm convert --set persistence.enabled=true stable/mongodb

Docker

You can also execute Helm convert from Docker:

$ docker run -ti containersol/helm-convert convert --help

Development

# clone the repo
$ git clone [email protected]:ContainerSolutions/helm-convert.git

# add a symlink in the Helm plugin directory targeting the repository
$ ln -s $PWD ~/.helm/plugins/helm-convert

# build the binary
$ make build

# run
$ helm convert --help

# run lint, vet and tests
$ make test-all

Features

The conversion is currently quite basic and has the following features:

• get image tags and store them in kustomization.yaml
• get common labels and store them in kustomization.yaml
• get resources and store them in kustomization.yaml
• remove helm specific labels from manifests
• remove helm specific annotations from manifests
• get namespace and store it in kustomization.yaml
• create secretGenerator based on secret resources (type Opaque and TLS)
• create secretGenerator based on secret type TLS
• create configGenerator from multiline files
• handle datasources type literal, env files and source files

helm-convert's Issues

Error using Docker image

I've got an error using the Docker image :

docker run -ti containersol/helm-convert convert --destination mongodb --name mongodb stable/mongodb
Error: no cached repo found. (try 'helm repo update'). open /root/.helm/repository/cache/stable-index.yaml: no such file or directory
Usage:
  convert [flag] [chart URL | repo/chartname] [...] [flags]

Examples:

  # convert the stable/mongodb chart
  helm convert --destination mongodb --name mongodb stable/mongodb

  # convert chart from a url
  helm convert https://s3-eu-west-1.amazonaws.com/coreos-charts/stable/prometheus-operator

  # convert the stable/mongodb chart with a given values.yaml file
  helm convert -f values.yaml stable/mongodb

  # convert the stable/mongodb chart and override values using --set flag:
  helm convert --set persistence.enabled=true stable/mongodb


Flags:
      --alsologtostderr                  log to standard error as well as files
      --ca-file string                   verify certificates of HTTPS-enabled servers using this CA bundle
      --cert-file string                 identify HTTPS client using this SSL certificate file
      --comments                         add default comments to kustomization.yaml file (default true)
      --dep-up                           run helm dependency update before installing the chart
  -d, --destination string               location to write the chart. If this and tardir are specified, tardir is appended to this
      --force                            convert chart even if the destination directory already exists
  -h, --help                             help for convert
      --home string                      location of your Helm config. Overrides $HELM_HOME (default "/root/.helm")
      --key-file string                  identify HTTPS client using this SSL key file
      --keyring string                   keyring containing public keys (default "/root/.gnupg/pubring.gpg")
      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
      --log_dir string                   If non-empty, write log files in this directory
      --logtostderr                      log to standard error instead of files (default true)
      --name string                      release name
      --namespace string                 global namespace to use for the manifests (default "default")
      --password string                  chart repository password
      --prov                             fetch the provenance file, but don't perform verification
      --repo string                      chart repository url where to locate the requested chart
      --set stringArray                  set values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)
      --set-file stringArray             set values from respective files specified via the command line (can specify multiple or separate values with commas: key1=path1,key2=path2)
      --set-string stringArray           set STRING values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)
      --skip-transformers strings        set a list of transformers that are skipped during the conversion process (can specify multiple or separate values with commas: secret,configmap)
      --stderrthreshold severity         logs at or above this threshold go to stderr (default 2)
      --username string                  chart repository username
  -v, --v Level                          log level for V logs
  -f, --values ValueFiles                specify values in a YAML file or a URL(can specify multiple) (default [])
      --verify                           verify the package against its signature
      --version string                   specific version of a chart. Without this, the latest version is fetched
      --vmodule moduleSpec               comma-separated list of pattern=N settings for file-filtered logging

Error: plugin "convert" exited with error

Show instructions when password are pulled from secret resources

Currently when a secret type Opaque is pulled from a secret (ie: password), it is converted in plain text and stored in the kustomization.yaml with a printf command, ie:

secretGenerator:
- commands:
    redis-password: printf \"OYTG9S4SUi\"
  name: sentry-redis
  type: Opaque

After the chart is deployed we need to warn the user that there is a plain text password stored there.

"release" label is not removed from Service selectors

The labels transformer removes the "chart", "release", and "heritage" labels from resources. However, it misses the ServiceSpec selector.

$ helm-convert https://kubernetes-charts.storage.googleapis.com/nginx-ingress-0.31.0.tgz
$ cat nginx-ingress/nginx-ingress-controller-svc.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    component: controller
  name: nginx-ingress-controller
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: http
  - name: https
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    app: nginx-ingress
    component: controller
    release: nginx-ingress
  type: LoadBalancer

Move from Travis to GitHub Actions

The current build & distribution is created by Travis. This currently has some problems, not least requiring uploading a personal access token to Travis CI.

It would be better and easier to move to GitHub actions. The build part should be trivial and creating distribution binaries shouldn't be much harder. At the same time the docker builds could be fixed and moved to ghcr.

Kustomize 4.0 support

Unfortunately with the major changes in Kustomize 4.0, this no longer generates valid KYaml. And as such, isn't entirely working.

Is this even still being maintained?

Improve formatting of the kustomization.yaml

Currently the kustomization.yaml file is hard to read when rendered. I would suggest separating each keys with an extra line break and add an option to add basic comments as listed here:
https://github.com/kubernetes-sigs/kustomize/blob/master/docs/kustomization.yaml

Image url with ports cause incorrect "newTag"

$ helm template exampleChart | grep 'image:'
          image: "my-registry:5000/namespace/app"
$ helm convert --destination example exampleChart
$ cat examples/kustomization.yaml | grep -A 3 images
# Images modify the tags for images without
# creating patches.
images:
- name: busybox
- name: my-registry
  newTag: 5000/namespace/app

Ability to exclude specific transformers

I find that I am always having to do some post-processing of the files generated by helm-convert. For example, I almost never want to use namePrefix and I often want to skip the configmap and secret transformers. It would be useful to have a way to specify a list of transformers to skip during the conversion process.

helm 3 support

Hi

This tool so promising to convert helm charts to regular Kubernetes native yaml files without using helm in some cases. I tried to use this tool today but getting below error. is this works for helm 3?

helm convert --destination mongodb --name mongodb stable/mongodb
Error: Couldn't load repositories file (/Users/skotaru/.helm/repository/repositories.yaml).
You might need to run helm init (or helm init --client-only if tiller is already installed)

Add configMapGenerator

Create a new transformer to pull configmap resources into configMapGenerator.

https://github.com/kubernetes-sigs/kustomize/blob/master/docs/kustomization.yaml#L64-L72

To be discussed, but the idea would be to pull multi-line entries and store them as file.

Error: chart not found in repository

Using this command

docker run -v $PWD:/kong -ti containersol/helm-convert:v0.5.3 convert --dep-up --name dev --namespace kong-dev --repo https://charts.konghq.com --destination /kong/kustomize kong/kong

throws an error

Error: chart "kong/kong/kong" not found in https://charts.konghq.com repository

Meanwhile helm repo add https://charts.konghq.com works fine on this container.

Was able to workaround this issue by specifying link to the chart itself:

docker run -v $PWD:/kong -ti containersol/helm-convert:v0.5.3 convert --dep-up --name dev --namespace kong-dev --destination /kong/kustomize https://github.com/Kong/charts/releases/download/kong-1.10.0/kong-1.10.0.tgz

Can not install through "helm plugin"

When installing I get this error:

abo@bofh42:~$ helm plugin install https://github.com/ContainerSolutions/helm-convert
sh: 1: /home/abo/.helm/plugins/helm-convert/install.sh: not found
Error: plugin install hook for "convert" exited with error

I'm on Helm 2.12.3 on an Ubuntu 18.04 AMD64 system with golang 1.10 installed.

helm-convert fails to output all resources

helm-convert does not output all resources if a helm chart has a single template that includes more than 1 resource.

For example, the cert-manager chart has a template rbac.yaml that includes a ClusterRole and a ClusterRoleBinding: https://github.com/helm/charts/blob/master/stable/cert-manager/templates/rbac.yaml

When you use helm-convert to convert this chart, the ClusterRoleBinding will not be generated:

$ ./helm-convert --force --name cm stable/cert-manager -d out                                                          
$ grep ClusterRole out/*                                                                                               
out/cm-cert-manager-clusterrole.yaml:kind: ClusterRole         

Installation broken on 0.5.1

helm plugin install https://github.com/ContainerSolutions/helm-convert fails because of the install hook:
https://github.com/ContainerSolutions/helm-convert/blob/v0.5.1/plugin.yaml#L11

specifically here:
https://github.com/ContainerSolutions/helm-convert/blob/v0.5.1/install.sh#L11

since https://github.com/ContainerSolutions/helm-convert/releases/download/v0.5.1/helm-convert_linux_0.5.1.tar.gz does not exist.

helm-convert does not support List resources

As an example, the prometheus-operator helm chart uses kind: List here: https://github.com/helm/charts/blob/master/stable/prometheus-operator/templates/prometheus/servicemonitors.yaml#L3

helm-convert fails on this chart:

$ helm-convert https://kubernetes-charts.storage.googleapis.com/prometheus-operator-0.1.25.tgz
2018/11/20 12:58:20 Error: no field named 'metadata.name'

Can't convert stunner-gateway-operator on github... gzip:invalid header

Hi, what's the best way to convert this helm chart? Thank you!

> $ docker run -ti containersol/helm-convert convert https://github.com/l7mp/stunner-helm/tree/main/helm/stunner-gateway-operator
Error: gzip: invalid header
Usage:
  convert [flag] [chart URL | repo/chartname] [...] [flags]

Examples:
...

'panic: interface conversion' when trying to convert oteemocharts/sonarqube

Attempted using the docker image or on my host machine running debian bullseye/sid. To reproduce:

$ helm repo add oteemocharts https://oteemo.github.io/charts
$ helm convert --destination /tmp/sonarqube --name sonarqube  oteemocharts/sonarqube
panic: interface conversion: interface {} is nil, not []interface {}

goroutine 1 [running]:
github.com/ContainerSolutions/helm-convert/pkg/transformers.(*imageTransformer).getImageTag(0xc0002bc740, 0xc0003cc160, 0xc000597ec0, 0xd0ef84, 0xe, 0x1, 0x0)
	/home/travis/gopath/src/github.com/ContainerSolutions/helm-convert/pkg/transformers/image.go:60 +0x513
github.com/ContainerSolutions/helm-convert/pkg/transformers.(*imageTransformer).findImage(0xc0002bc740, 0xc0003cc160, 0xc000597ec0, 0x4, 0xc000214998)
	/home/travis/gopath/src/github.com/ContainerSolutions/helm-convert/pkg/transformers/image.go:47 +0x16d
github.com/ContainerSolutions/helm-convert/pkg/transformers.(*imageTransformer).findContainers(0xc0002bc740, 0xc0003cc160, 0xc000597e30, 0xe, 0x143c060)
	/home/travis/gopath/src/github.com/ContainerSolutions/helm-convert/pkg/transformers/image.go:109 +0x20c
github.com/ContainerSolutions/helm-convert/pkg/transformers.(*imageTransformer).findImage(0xc0002bc740, 0xc0003cc160, 0xc000597e30, 0x8, 0xc0002142f8)
	/home/travis/gopath/src/github.com/ContainerSolutions/helm-convert/pkg/transformers/image.go:54 +0x100
github.com/ContainerSolutions/helm-convert/pkg/transformers.(*imageTransformer).findContainers(0xc0002bc740, 0xc0003cc160, 0xc000597da0, 0xe, 0x143c060)
	/home/travis/gopath/src/github.com/ContainerSolutions/helm-convert/pkg/transformers/image.go:109 +0x20c
github.com/ContainerSolutions/helm-convert/pkg/transformers.(*imageTransformer).findImage(0xc0002bc740, 0xc0003cc160, 0xc000597da0, 0x4, 0xc0001b7f18)
	/home/travis/gopath/src/github.com/ContainerSolutions/helm-convert/pkg/transformers/image.go:54 +0x100
github.com/ContainerSolutions/helm-convert/pkg/transformers.(*imageTransformer).findContainers(0xc0002bc740, 0xc0003cc160, 0xc000597d10, 0xe, 0x143c060)
	/home/travis/gopath/src/github.com/ContainerSolutions/helm-convert/pkg/transformers/image.go:109 +0x20c
github.com/ContainerSolutions/helm-convert/pkg/transformers.(*imageTransformer).findImage(0xc0002bc740, 0xc0003cc160, 0xc000597d10, 0xc000267f68, 0x0)
	/home/travis/gopath/src/github.com/ContainerSolutions/helm-convert/pkg/transformers/image.go:54 +0x100
github.com/ContainerSolutions/helm-convert/pkg/transformers.(*imageTransformer).Transform(0xc0002bc740, 0xc0003cc160, 0xc0001e4010, 0x0, 0x0)
	/home/travis/gopath/src/github.com/ContainerSolutions/helm-convert/pkg/transformers/image.go:28 +0x12c
github.com/ContainerSolutions/helm-convert/pkg/transformers.(*multiTransformer).Transform(0xc0002bc760, 0xc0003cc160, 0xc0001e4010, 0x8, 0xc0005e5b28)
	/home/travis/gopath/src/github.com/ContainerSolutions/helm-convert/pkg/transformers/multitransformer.go:27 +0x79
github.com/ContainerSolutions/helm-convert/cmd.(*convertCmd).run(0xc00018c180, 0xc0000fbd88, 0xc0000fbd58)
	/home/travis/gopath/src/github.com/ContainerSolutions/helm-convert/cmd/convert.go:255 +0xc3b
github.com/ContainerSolutions/helm-convert/cmd.NewConvertCommand.func2(0xc00014a280, 0xc0000b0c80, 0x1, 0x5, 0x0, 0x0)
	/home/travis/gopath/src/github.com/ContainerSolutions/helm-convert/cmd/convert.go:106 +0x99
github.com/spf13/cobra.(*Command).execute(0xc00014a280, 0xc0000321f0, 0x5, 0x5, 0xc00014a280, 0xc0000321f0)
	/home/travis/go/pkg/mod/github.com/spf13/[email protected]/command.go:762 +0x465
github.com/spf13/cobra.(*Command).ExecuteC(0xc00014a280, 0xd63ee0, 0xc0000fbf88, 0xc0000380b8)
	/home/travis/go/pkg/mod/github.com/spf13/[email protected]/command.go:852 +0x2ec
github.com/spf13/cobra.(*Command).Execute(...)
	/home/travis/go/pkg/mod/github.com/spf13/[email protected]/command.go:800
main.main()
	/home/travis/gopath/src/github.com/ContainerSolutions/helm-convert/main.go:13 +0x44
Error: plugin "convert" exited with error

defaultMode: 0755 gets overriden by defaultMode: 493 after converting, breaking scripts in ConfigMaps

Can be seen using the bitnami/mongodb chart.

helm repo add bitnami https://charts.bitnami.com/bitnami
helm convert --name mongodb --set architecture=replicaset bitnami/mongodb
cd mongodb
grep 'defaultMode' mongodb-statefulset.yaml

> defaultMode: 493

This unfortunately breaks the mongodb chart from working, as the script is used as the container command and cannot be run this way due to permissions.

stable/prometheus/operator chart throw error: failed to read Resources: id

Investigate what is causing the following message to occur:

$ helm convert stable/prometheus-operator --version 4.3.5 
$ kustomize build prometheus-operator
Error: rawResources failed to read Resources: id '"rbac.authorization.k8s.io_v1_RoleBinding|default|~P|prometheus-operator-prometheus|~S"' already used

Improve labels handling

Currently, we remove labels based on a list of YAML keys which could break the initial chart. For example, if a custom resource definition contains a key with the same name as provided in the list to remove and this key has nothing to do with labels, then it will break:

apiVersion: custom/v1
kind: MyCustomResource
metadata:
  name: my-custom-resource
spec:
  something:
    labels: # << this shouldn't be removed but helm-convert will remove it
      someConfig: someValue
      someMoreConfig:
        labels:  ## << this should be removed
           some: label
        someMoreNestedConfig: someValue

To prevent this to happen, we could create a map of safe path per resource using the following format which could be parsed into a GVK type <api group>/<api version>/<kind>:

paths := map[string][]string{} {
  "custom/v1/MyCustomResource": []string{
    "spec.something.labels.someMoreConfig.labels",
  },
}

By doing so we would need to provide the path of all the resources and make it possible to pass custom path via the CLI.

For natives resources (apps/v1, policy/v1beta1, etc.). We can keep the current behavior.