kubecd / kubecd

Kubernetes Continuous Deployment Toolkit

Kubernetes Continuous Deployment Tool

kubecd is a deployment tool for Kubernetes that lets you declare in Git what should be deployed in all your environments, manage image upgrade strategies per service, and make it so. It supports any Kubernetes installation with some help, but has direct support for GKE or Azure, minikube and Docker, and deployment using Helm or plain kubectl.

Goals

• Provide a mechanism for defining "environments", which are namespaces in clusters, where developers deploy their apps
• Allow a layered config mechanism to share environment-specific values among Helm charts deployed in the environment (for things like Ingress domain)
• Build as much as possible on top of existing mainstream tools (kubectl and helm)
• Support a GitOps workflow by providing a tool with little opinion included, allowing you to assemble your pipeline to suit your needs.

Configuring Environments

All the deployable environments are configured in a file typically called environments.yaml. The schema for this file can be found here (check the KubeCDConfig struct).

This file must contain two sections/keys, clusters and environments. Each environments maps to one namespace in one cluster, but the environment names must be unique within this file.

Example:

helmRepos:
  - name: stable
    url: https://kubernetes-charts.storage.googleapis.com/

clusters:
  - name: prod-cluster
    provider:
      gke:
        project: example-com-prod
        clusterName: prod-cluster
        zone: us-central1-c
  - name: test-cluster
    provider:
      gke:
        project: example-com-test
        clusterName: test-cluster
        zone: us-central1-c

environments:
  - name: prod
    clusterName: prod-cluster
    kubeNamespace: default
    releasesFiles:
      - common/base-env.yaml
      - prod/releases.yaml
    defaultValues:
      - key: "image.prefix"
        value: "gcr.io/example-com-prod/"
      - key: "ingress.domain"
        value: "prod.example.com"
  - name: test
    clusterName: test-cluster
    kubeNamespace: default
    releasesFiles:
      - common/base-env.yaml
      - test/releases.yaml
    defaultValues:
      - key: "image.prefix"
        value: "gcr.io/example-com-test/"
      - key: "ingress.domain"
        value: "test.example.com"

Here, we have defined two environments, test and prod, each running in separate GKE clusters in different GCP projects. We have also set some default helm chart values which will be automatically applied to every chart deployed into those environments, so that if your chart uses the ingress.domain value to construct the full Ingress host, you do not have to worry about specifying or overriding that domain part in every single release/deployment.

Configuring Releases

Once you have your environments defined, you need to configure what should be deployed into each of them. This is expressed as "releases" (term borrowed from Helm).

A "releases" file contains a list of releases, for example:

releases:
  - name: ingress
    chart:
      reference: stable/nginx-ingress
      version: 1.15.0
    valuesFile: values-ingress.yaml

See more examples here: releases-common.yaml, releases-prod.yaml, releases-test.yaml.

Installing and Running

To produce a kcd binary:

make build

Contributing

When submitting PRs, please ensure your code passes gofmt, go vet and go test.

For bigger changes, please create an issue proposing the change in advance.

Design Document

Can be found here.

kubecd's Issues

Open source the project

Checklist:

• Squash commits
• Remove internal Zedge references
• Migrate CI build to Travis or Circle

Support for --debug

Please support passing '--debug' in to helm :)

Container Registry notifications

Add an interface for receiving image update notifications from registries, and implementations for:

• docker registry v2 webhooks
• docker hub webhooks
• GCR pubsub notifications (both as pubsub and with push urls)
• ACR webhooks
• quay.io webhooks

kcd diff does not work

$ kcd diff
Traceback (most recent call last):
  File "/home/steineldar/.local/bin/kcd", line 11, in <module>
    sys.exit(main())
  File "/home/steineldar/.local/venvs/kubecd/lib/python3.6/site-packages/kubecd/cli/__init__.py", line 291, in main
    func(**kwargs)
  File "/home/steineldar/.local/venvs/kubecd/lib/python3.6/site-packages/kubecd/cli/__init__.py", line 93, in diff_release
    e = kcd_config.env_by_name(env)
  File "/home/steineldar/.local/venvs/kubecd/lib/python3.6/site-packages/kubecd/model.py", line 142, in env_by_name
    return self._env_index[name]
KeyError: None

New environment should create namespace if it doesn't exist

Currently you have to invoke the kubectl create namespace foo if namespace foo doesn't exist and you've configured up an environment to use a given namespace foo.

End user expects the namespace to be created automatically since kubecd is a tool for declarative defining environments and to apply a whole environment without manual interaction.

apply -r a -r b only applies last

The argument accept multiple instances, but only applies the last one.

kcd apply test -r first -r last

only applies last.

Do not allow duplicate registration of releases

We do not want to allow register the release foo multiple times in the same environment.

kcd should panic and give an early error.

Exception when using --release with poll.

$ kcd poll test -r test/test-service -p
Traceback (most recent call last):
  File "/home/steineldar/.local/bin/kcd", line 11, in <module>
    sys.exit(main())
  File "/home/steineldar/.local/venvs/kubecd/lib/python3.6/site-packages/kubecd/cli/__init__.py", line 243, in main
    func(**kwargs)
  File "/home/steineldar/.local/venvs/kubecd/lib/python3.6/site-packages/kubecd/cli/__init__.py", line 141, in poll_registries
    logger.info('polling release: "%s/%s"', environment.name, release_obj.name)
AttributeError: 'NoneType' object has no attribute 'name'

Add policy framework

Create a model for validation policies, to be applied against pre-rendered API objects before running "apply". Policies could include:

• Only one Ingress should have the same host/path combination
• All helm-rendered resources must have the standard labels (chart, release, heritage)
• Every workload (Deployment/StatefulSet etc.) should be using a minimum (or specific) apiVersion (with exceptions)
• metadata.namespace should never be set explicitly (with exceptions)

The policy model should be flexible enough (or extensible) to allow users to declare their own policies.

kcd apply prod -r does-not-exist is OK

should complain if trying to apply / poll etc a non-existing release.

Proper error message if in wrong directory.

In stead of stacktrace.

Traceback (most recent call last):
  File "/home/steineldar/.local/bin/kcd", line 11, in <module>
    sys.exit(main())
  File "/home/steineldar/.local/venvs/kubecd/lib/python3.5/site-packages/kubecd/cli/__init__.py", line 236, in main
    func(**kwargs)
  File "/home/steineldar/.local/venvs/kubecd/lib/python3.5/site-packages/kubecd/cli/__init__.py", line 102, in dump_env
    target_envs = one_or_all_envs(env, all_environments, file_name=config_file)
  File "/home/steineldar/.local/venvs/kubecd/lib/python3.5/site-packages/kubecd/cli/__init__.py", line 187, in one_or_all_envs
    load_envs(file_name)
  File "/home/steineldar/.local/venvs/kubecd/lib/python3.5/site-packages/kubecd/cli/__init__.py", line 180, in load_envs
    model.load(file_name)
  File "/home/steineldar/.local/venvs/kubecd/lib/python3.5/site-packages/kubecd/model.py", line 202, in load
    _config = KubeCDConfig(load_yaml_with_schema(file_name, ttypes.KubeCDConfig), from_file=file_name)
  File "/home/steineldar/.local/venvs/kubecd/lib/python3.5/site-packages/kubecd/thriftutils.py", line 5, in load_yaml_with_schema
    with open(yaml_file, 'r') as fd:
FileNotFoundError: [Errno 2] No such file or directory: 'environments.yaml'

Support namespaced tiller

Right now, kubecd assumes that Helm's Tiller runs in kube-system. Each environment should be able to specify whether it's using its own namespaced Tiller, for example like this:

  - name: prod
    clusterName: demo-cluster
    kubeNamespace: prod
    tillerNamespace: prod

All that's needed is a small change in the generate_helm_base_argv() function in kubecd/helm.py.

EKS support

Add a provider for Amazon EKS

Get rid of `gcloud` command-line dependency

The gcloud command is part of google-cloud-sdk, which is several hundred megabytes.
It's currently used for two things, getting cluster credentials and looking up static IP addresses.
Both should be doable through the Python SDK, possibly with some manual fiddling around with kubeconfig to inject credentials.

`kcd observe` should have a flag to limit to environment

kcd observe with non-updated helm repo prints stack strace

$ kcd observe --image us.gcr.io/my-project/my-service:abcdef0 --patch
Error: failed to download "stable/redis-ha"
Traceback (most recent call last):
  File "/home/steineldar/.local/bin/kcd", line 11, in <module>
    sys.exit(main())
  File "/home/steineldar/.local/lib/python3.6/site-packages/kubecd/cli/__init__.py", line 175, in main
    func(**kwargs)
  File "/home/steineldar/.local/lib/python3.6/site-packages/kubecd/cli/commands.py", line 120, in observe_new_image
    image_index = kcd_config.image_index
  File "/home/steineldar/.local/lib/python3.6/site-packages/kubecd/model.py", line 194, in image_index
    values = helm.get_resolved_values(rel, env, skip_value_from=True)
  File "/home/steineldar/.local/lib/python3.6/site-packages/kubecd/helm.py", line 224, in get_resolved_values
    output = inspect(release.chart.reference, release.chart.version)
  File "/home/steineldar/.local/lib/python3.6/site-packages/kubecd/helm.py", line 20, in inspect
    output = subprocess.check_output(cmd).decode('utf-8').split("\n---\n")[1]
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['helm', 'inspect', 'stable/redis-ha', '--version', '2.2.3']' returned non-zero exit status 1.

Should print shorter error and suggest to update helm.