spinnaker / kleat

A lightweight tool for managing Spinnaker configuration

kleat

Kleat is tool for managing Spinnaker configuration files that is intended to replace Halyard. Please see the RFC for an overview of what this tool will do.

It is currently under active development, but is ready for early adopters to try in development clusters! Please share any feedback in the #kleat channel in Spinnaker slack.

Please see spinnaker/kustomization-base for step-by-step instructions on how to install Spinnaker to a Kubernetes cluster using kleat and kustomize.

Migrating from Halyard

Please see the migration README for details on migrating from Halyard.

Installation

Download the appropriate binary from the releases page.

Usage

• kleat generate /path/to/halconfig /path/to/output/directory
• kleat help
• kleat version

Philosophy

The overarching design goal of kleat is that is should be as lightweight as possible, and should know as little as possible about implementation details of Spinnaker's microservices.

Problems should be solved at the right level of abstraction. For example:

• If a microservice does not set reasonable defaults, and requires a lot of config to be usable, this should be fixed by having the microservice set reasonable defaults rather than by having kleat hard-code defaults.
• If configuration errors do not lead to useful error messages, the microservice shoiuld be updated to provide useful error messages instead of relying on kleat to catch erroneous config.

This is in contrast to Halyard, which often had detailed knowledge of how the Spinnaker microservices worked (even to the point of importing a number of the microservices as libraries).

Much of the the effort in writing kleat involved:

• Updating the microservices to set sensible defaults, so that these did not need to be set in kleat
• Strongly typing and documenting the fields in the halconfig and the microservice configs

Defaulting

All configurable fields should have sensible defaults; users should be able to get a working installation of Spinnaker with as little config as possible.

The microservices are responsible for defaulting config properties to sensible values; kleat should not explicitly default any config properties. If a property has not been explicitly configured by the user, kleat should omit it from the generated output and let the relevant microservice handle defaulting.

While it might seem reasonable to have kleat set default values for a service's properties, the need to do this is generally a sign that the service should better default its properties.

Our experience with Halyard is that having a config management tool handle defaulting leads to the following issues:

• The config management tool needs to understand too much about the implementation details of the microservices it is deploying, breaking an important layer of abstraction.
• Users who deploy manually, without a config management tool, don't get the benefit of these defaults and still need to manually write a lot of config.
• The microservice is unable to distinguish values that have been explicitly set by the user from values that were defaulted by the config management tool. This makes it difficult to add or change defaults at the microservice level.
• The config generated by the tool becomes less readable, as it contains a lot of default values that are not of interest to the user, instead of only values that the user has explicitly configured.

One of the reasons that Halyard became difficult to maintain is that it tried to solve a lot of problems that were better solved at a different level of abstraction.

Transformation of values

The core competency of kleat is understanding what parts of the halconfig belong in each microservice's config. This is valuable, as:

• End users do not need to understand what each microservice does, and can just configure their halconfig and defer to kleat to generate microservice configs.
• Some config parameters (such as account credentials) are used by more than one microservice; kleat allows users to specify these parameters once and handles duplicating them in the required config files.

kleat should not transform input fields to the halconfig and should just pass these through unchanged to the appropriate places in the output configs.

If it seems necessary for kleat to transform a config value before passing it along to the microservice, please consider updating the relevant microservice to handle this transformation when loading its config. This will allow users manually writing their microservice configs to benefit from the change, and will keep the code handling this transformation in the relevant microservice rather than in the config management tool.

kleat's Issues

Kleat doesn't fill in echo.yml's Spinnaker baseUrl

When Kleat generates the echo.yml file, it should additionally generate the field spinnaker.baseUrl which it might be okay to infer from security.uiSecurity.overrideBaseURL in the halconfig.

The result should be something like this in the head of echo.yml:

spinnaker:
  baseUrl: https://spinnaker.my.domain

For more context. Slack thread.

Thanks @nimakaviani and @kskewes for your help.

The SecurityContext in deck deployment results in the deck container crashing

Looks like the new securityContext introduced for a deck deployment here breaks it for me. I got the following error when trying to deploy deck and had to disable the security context for it to work. thoughts?

cp: cannot create regular file 'spinnaker.conf': Permission denied
sed: can't read spinnaker.conf: No such file or directory
sed: can't read spinnaker.conf: No such file or directory
sed: can't read spinnaker.conf: No such file or directory
mv: cannot stat 'spinnaker.conf': No such file or directory
ERROR: Site spinnaker does not exist!
Could not remove /etc/apache2/sites-enabled/000-default.conf: Permission denied
cp: cannot create regular file 'ports.conf': Permission denied
sed: can't read ports.conf: No such file or directory
sed: can't read ports.conf: No such file or directory
mv: cannot stat 'ports.conf': No such file or directory
cp: cannot create regular file 'passphrase': Permission denied
sed: can't read passphrase: No such file or directory
chmod: cannot access 'passphrase': No such file or directory
mv: cannot stat 'passphrase': No such file or directory
cp: cannot create regular file '/opt/deck/html/settings.js': Permission denied
chown: changing ownership of '/var/lock/apache2.g9wrlkWKEk': Operation not permitted```

Support versioning of kustomization-base

Proposal

It would be better to use Git tag's versioning in kustomization-base for the users.

kustomize has already supported the use of Git refs so all we need to do is to create Git tags.

Related repositories

• spinnaker/kustomization-base

Background

kleat would work well with kustomization-base, though it's not necessary. So I guess many kleat's users would use the base kustomization.

Currently, the users need to specify bases in this way, in order to use kustomization-base:

bases:
- github.com/spinnaker/kustomization-base/core

This way might break the user's configurations because it always references master (or main) branch so the user's configurations will be affected if kustomization-base is broken.

With versioning like below:

bases:
- github.com/spinnaker/kustomization-base/core?ref=v1.2.3

It allows users to use the specific version with conviction, after examining if the version works with their configurations.

References

• kustomization.yaml · The Kubectl Book

Kleat missing default Kubernetes account in Deck's "settings.json"

What

When kleat generates Deck's setting.js, it is missing Kubernetes default configuration.

Which version

• kleat: 0.4.0
• Spinnaker: 1.20.8

How to reproduce

$ kleat generate <any_halconfig>
(see `settings.json`)

kleat will generate a file like this.

window.spinnakerSettings = JSON.parse('{"gateUrl":"https://spinnaker-api.something.domain.com","authEnabled":true,"authEndpoint":"https://spinnaker-api.something.domain.com/auth/user","bakeryDetailUrl":"https://spinnaker-api.something.domain.com/bakery/logs/{{context.region}}/{{context.status.resourceId}}","canary":{"defaultJudge":"NetflixACAJudge-v1.0","featureDisabled":false,"showAllConfigs":true,"templatesEnabled":true},"notifications":{"slack":{"enabled":false,"botName":"spinnakerbot","token":"${SLACK_TOKEN}"}},"providers":{"kubernetes":{}},"defaultTimeZone":"America/Los_Angeles","feature":{"pipelineTemplates":true,"chaosMonkey":false,"fiatEnabled":true,"managedPipelineTemplatesV2UI":true}}');

Here, the provider.kubernetes is {}. But it should be provider.kubernetes: {default:{}} to define the default property which the deck requires. If I apply the manifests generated by kleat, it will run into error in the deck.

Workaround

I'm currently configuring this manually is settings-local.json.

$ echo "window.spinnakerSettings.providers.kubernetes = { defaults: {} }" >> settings.json

How to overriding of the service discovery file for fiat service.

From here, to enable Fiat, need to configure a service discovery overrides file in the fork of spinnaker-config.

But I couldm't find a way to configure the overrides. Could someone please provide some help on this.

`kleat generate` expects that the output dir already exists

If kleat generate is invoked with a dir an output dir that doesn't exist, you get the following error.

Error: stat ./generated: no such file or directory

Proposed solutions:

1. Notify the user that they need to create that directory
2. Make that directory for the user
3. Something else I didn't think of?

Which would you prefer in the PR?

Add support for installing Keel

Keel is the microservice required to enable Managed Delivery for Spinnaker. As part of our efforts to make Managed Delivery available to the larger community, we would like to have Kleat generate the config file for Keel and also to add the required adjustments to the config files for other Spinnaker microservices.

Here is a PoC implementation: https://github.com/nimakaviani/kleat

The implementation introduces a feature flag --enable-keel that once set, makes the required config changes. From our discussions in the k8s sig meeting, @ezimanyi suggested we might be able to remove the feature flag and instead pre-populate the config files with acceptable defaults.

Any other changes to the above implementation that you see appropriate?

We can start working on the PR once we settle on the final design. thanks!

/cc @queueburt

Can't set trustStore and trustStorePassword for JenkinsAccount

We're transforming to use kleat to deploy our spinnaker.
Currently, we configure trustStore and trustStorePassword for some of our jenkins masters.
ci: jenkins: enabled: true masters: - name: pc-cpc permissions: {} address: https://jenkins.url/ username: spinn-user password: xxxxxxxxxxxxxxxx csrf: false trustStore: /home/spinnaker/.hal/production/staging/dependencies/1176791443-EGADIssuingCA3.jks trustStorePassword: cxiSqS5NFxxxxxx

But there're no trustStore and trustStorePassword in the kleat config page. https://github.com/spinnaker/kleat/blob/master/docs/docs.md#proto.ci.JenkinsAccount

SQL as the backing datastore for Spinnaker microservices

Spinnaker microservices can be configured to use SQL instead of Redis as the backing datastore. This issue serves as a place to track changes to be made to the SQL proto config as well as the respective Spinnaker microservices for SQL support.

SQL configuration is introduced to Kleat in #143 to enable it for Keel but it requires adding extra properties for it to work with other microservices.

Gate sends traffic to clouddriver-rw instead of clouddriver-ro-deck

When using the kleat + kustomize deployment path, traffic from gate gets sent to clouddriver-rw instead of clouddriver-ro-deck.

This is because there is no override in the gate config file; traffic just goes to the generic clouddriver service, which points to clouddriver-rw. We'll likely need to add an override in the gate config (or come up with a more generic solution).

Basic auth settings do not seem to generate a correct Gate config

I'm trying to set up basic auth for tools to interact with Gate without needing a heavier auth flow. Following the existing docs, I have

security:
  authn:
    enabled: true
    basic:
      enabled: true
      user:                                                                                   
        username: basic-user
        password: basic-password

Running the latest kleat generates a gate.yml file with the following:

security:                                                                                                                                     
  basic:                                                                                                                                      
    enabled: true                                                                                                                             
    user:                                                                                                                                     
      password: basic-user                                                                                                                 
      username: basic-password

However, Gate itself does not seem to accept this format; from the logs:

2021-09-29 18:24:54.449 ERROR 1 --- [           main] o.s.b.c.p.m.PropertiesMigrationListener  : 
The use of configuration keys that are no longer supported was found in the environment:

Property source 'applicationConfig: [file:/opt/spinnaker/config/gate.yml]':
        Key: security.basic.enabled
                Line: 3
                Reason: The security auto-configuration is no longer customizable.


Please refer to the migration guide or reference guide for potential alternatives.

From looking at Gate's test suite, it seems a basic auth setup might look more like

security.basicform.enabled=true
spring.security.user.name=basic-user
spring.security.user.password=basic-password

and it's possible this changed at 70024a3.

Am I correct in thinking that changes need to be made to kleat to support Gate's newer basic auth configuration format?