zbynek / charts

Jenkins Infrastructure Helm Charts

Jenkins Infrastructure Helm Charts

This project is still heavily in development

This repository defines and stores helm charts used by jenkins infrastructure project to configure its clusters.

It also combines a Jenkinsfile, helmfiles and helm charts to automate Kubernetes Cluster configuration.

Repository Structure

This project contains three main folders:

• helmfile.d: This folder contains Helmfile
• charts: This folder contains specific jenkins infrastructure helm charts
• config: The configuration specific our environments.

Secrets

Secrets are encrypted with sops and a default configuration is defined in .sops.yaml Currently there are two keys, one GPG key and a second one in an azure key vault and accessible from Kubernetes clusters.

In order to edit a secret, just run sops <your yaml file>

Docker

This folder defines a custom Dockerfile in order to build a custom image to orchestrate our clusters

Rules

• Secrets are define in secrets.yaml file and always encrypted

Remarks

• When deploying nexus, the postStart strict hang from time to time and we have to manually execute while containers are in creating mode kubectl exec -i -t -c nexus default-release-nexus-0 /opt/sonatype/nexus/postStart.sh

• We need one jenkins instance per cluster so we should split cluster orchestration tasks outside release.ci.jenkins.io

• If RBAC is enabled on the cluster, before being able to use helm, we need to create a service acccount for helm with the right cluster role binding. we can run following command: kubectl apply -f helm/rbac.yaml

Links

• Helmfile
• Helm Charts
• Sops