Coder Social home page Coder Social logo

omegasquad82 / concourse-k0sctl Goto Github PK

View Code? Open in Web Editor NEW
2.0 2.0 0.0 471 KB

Containerimage and Concourse-ci pipeline to manage k0s clusters with k0sctl by https://k0sproject.io

License: MIT License

Dockerfile 20.64% Shell 79.36%
concourse-pipeline container-image k0s kubernetes

concourse-k0sctl's Introduction

concourse-k0sctl

Codacy Badge

It's a container image which instruments k0sctl, a CLI to manage k0s Kubernetes clusters. The image currently is publicly available on Docker Hub as omegasquad82/k0sctl-handler. This project aims to provide a simple, configurable Concourse pipeline example, but it is neither production ready nor currently intended to be run in security sensitive settings. It may improve over time. Feedback, ideas and any other contribution is welcome.

containerimage

The image built with the Dockerfile is based on Alpine Linux 3.16.x and the buildx ci workflow has been prepared to build it both for linux/amd64 and linux/arm64 targets. Currently there is a lack in smoke testing during the build and testing unfortunately done via the pipeline template. Until this has been corrected it still occasionally breaks during tinkering.

security

Both pipeline and containerimage are being built in my free time and are a fun project. In contrary to what the above states it is imperative that you never leak any private key data you handle. This product encrypts the files created with k0sctl backup. This feature cannot be turned off at this point in time.

It is required that you generate, and provide via pipeline vars, a cluster.gpg_pair with the exact same Name and Mail, but no Comment, values you're providing to the pipeline as cluster.name and cluster.email.

These values will be used to both en- and later decrypt the backup password as well as sign all commits to the backup git repository during relevant Jobs. Please review this document for more information about commit signature verification.

praise

My gratitude to @rstacruz as I'm heavily relying on their cheat sheets for both bash and markdown during my day to day work.

inventory

Please see the Dependency Dashboard for a full inventory of automatically maintained versions for Github Actions, Dockerfile and Packages.

Alpine

package
bash
coreutils
curl
git
gnupg
grep
mtr
openssl

Github

package
k0sctl

scripts

A few functions used either in the image or the pipeline or both.

It is the main glue between the pipeline and the CLI. It has several environment parameters, listed in the order of appearance:

Name Description Default
K0SCTL_CMD_NAME The action to perform. version
DISABLE_TELEMETRY Pipeline's no_telemetry false
K0SCTL_SSH_KEY private SSH key content
K0SCTL_SSH_TYPE the key's file name id_ed25519
K0SCTL_CFG_PATH to k0sctl config spec config/k0sctl.yaml
K0SCTL_DIR_LOG to store k0sctl's log auditlog
K0SCTL_DIR_BAK to place backups into. backup
K0SCTL_DIR_RES from where to restore restore
K0SCTL_GPG_KEY to decrypt backup password
K0SCTL_ENC_CIPHER openssl cipher for backups chacha20
K0SCTL_PREFIX_BAK Prefix of backup archives k0s_backup
K0SCTL_SUFFIX_LOG Final logfile's suffix log
K0SCTL_LOG_PATH k0sctl default log path ~/.cache/k0sctl/k0sctl.log
K0SCTL_SUFFIX_BAK Suffix of backup archives tar.gz

Pipeline

You'll find it's specification in pipeline.yml. k0sctl pipeline

Variables

An example parametrization is in var-example.yml.

path concourse resource type description
email pcfseceng/email-resource email alerting parameters
timer.ping time when to execute traceroutes
timer.backup time when to execute backups

k0sctl

Below the k0sctl parameter structure you'll find:

path concourse type description
config git place to fetch k0sctl configuration
backup git to backup/restore the cluster state
cluster._key string private SSH key
cluster.gpg_pair string private GPG key
cluster.mail string committer's email address for backups
cluster.name string email alert subject preamble
flags.no_telemetry boolean wether k0sctl should call home

Jobs

ping

It will periodically traceroute using mtr any valid IPv4 address present in the files retrieved by config/*.yaml glob.

init

It initializes the backup git repository with an encrypted secret.gpg file that contains a password to safely store the backups. An existing backup branch will be overwritten on subsequent builds.

the init Job created a branch with an encrypted password

install

This Job calls k0sctl apply with configuration from the config repository under it's default path k0sctl.yaml (configurable). If a non-empty k0sctl_backup_latest file exists in the backup repository, it will be decrypted with openssl enc handed over to k0sctl, which will restore the cluster's state if and only if it is a new installation.

k0sctl restored the cluster state

uninstall

Destroys the cluster by calling k0sctl reset.

backup

Calls k0sctl backup and encrypts it's output archive with openssl enc and the encrypted file will be saved in the backup git repository. A symlink will be created to easily access it during the restore operation.

k0sctl backup archives


concourse-k0sctl's People

Contributors

dependabot[bot] avatar omegasquad82 avatar renovate[bot] avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar

concourse-k0sctl's Issues

Separate restore Job is superfluous for existing clusters

As the restore Job relies on executing 'k0sctl apply --restore-from=' it would install the cluster with the values from the k0sctl.yaml but restore the cluster's state if and only if the cluster has been reset upfront. Judging by the initial and current restore.go source it seems that the module would only proceed if there is no running installation. It therefore seems unnecessary to have a separate restore Job when it's functionally equivalent to the install job employing 'k0sctl apply' but without any prior backups.

  • in k0sctl-handler.sh move the restore functionality to the install part
  • add the backup resource to the install Job in the pipeline
  • remove the restore Job from the pipeline

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

dockerfile
Dockerfile
  • docker/dockerfile 1.4
  • alpine 3.18.3
github-actions
.github/workflows/anchore.yml
  • actions/checkout v4
  • anchore/scan-action 966ad43c0c7aa23b622e7605e9c48be50da4dbc2
  • github/codeql-action v2
.github/workflows/buildx-ci.yml
  • actions/checkout v4
  • docker/setup-buildx-action v3
  • docker/login-action v3
  • docker/build-push-action v5
  • docker/build-push-action v5
.github/workflows/codacy.yml
  • actions/checkout v4
  • codacy/codacy-analysis-cli-action 3b66437c0b315d8b5da4c2471860f7377c3dbcd4
  • github/codeql-action v2
pre-commit
.pre-commit-config.yaml
  • pre-commit/pre-commit-hooks v4.4.0
  • pre-commit/mirrors-prettier v3.0.3
  • jumanjihouse/pre-commit-hooks 3.0.0
  • hadolint/hadolint v2.12.0
regex
Dockerfile
  • alpine_3_17/bash 5.2.15-r0
  • alpine_3_17/coreutils 9.1-r0
  • alpine_3_17/curl 8.3.0-r0
  • alpine_3_17/git 2.38.5-r0
  • alpine_3_17/gnupg 2.2.40-r0
  • alpine_3_17/grep 3.8-r1
  • alpine_3_17/k0sctl 0.14.0-r6
  • alpine_3_17/mtr 0.95-r1
  • alpine_3_17/openssl 3.0.11-r0

  • Check this box to trigger a request for Renovate to run again on this repository

Backups are unencrypted

Currently the backup archives are directly pushed to git w/o prior encryption. Implement a scheme derived from the k0sproject.io example

  • extend the Dockerfile by installing the gnupg
  • extend the Pipeline's parameters by taking a GPG key.
  • duplicate the printHeading as printFunction with less visual intrusiveness and shorter length.
  • create function prepareGPG() to import a gpg_pair into it's configuration.
  • create a function mkpw() which uses graphical characters and 24 chars default length
  • call it in the init Job and encrypt the password with said GPG key, store it as file secret.gpg.
  • use the secret and openssl enc to encrypt a new backup
  • dynamically decrypt the 'secret.gpg' file from the backup repository and use it to decrypt a backup to be restored

Dependency Management for Alpine packages

Create configuration and comments for @renovatebot's [repology-datasource|https://docs.renovatebot.com/modules/datasource/#repology-datasource] on all versioned packages in the Dockerfile

  • make Repology known to renovate.json
  • pull out all versions to environment
  • create a repology comment on variables
  • test if this works within the branch

Create proper smoke tests

Currently the image is being tested via an instantiated pipeline in my private Concourse installation. There should be one for every step callable by k0sctl-handler.sh.

  • version
  • install
  • uninstall
  • backup

CI is cluttered

Please check which of the code scanning is really necessary and consolidate it into the buildx-ci.yml!

Documentation is missing

There is no documentation, provide meaningful descriptions of the pipeline, an example parameterization and screenshots.

  • redesign the parameter interface
  • provide an example var-file
  • create descriptions
  • create screenshots

SSH keys for control plane and data plane are shared

We have some trust in the security of Concourse's backing secrets store, however sharing the key pair between the controllers and workers can be considered unsafe in case either private key has been leaked. At least two sets of SSH keys should be used when managing clusters. In extremo one key per node might be desirable, which is out of scope for this issue.

  • add k0sctl.ssh data structure
  • let every element of this be a name:keydata attribute
  • create a new task prepare-ssh-keys to write from the data structure to it's ssh output
  • create a new input ssh to the k0sctl-task to propagate keys to the handler
  • update the k0sctl-handler by adding K0SCTL_SSH_DIR variable
  • update the k0sctl-handler with code that copies the contents of the K0SCTL_SSH_DIR to the ~/.ssh directory
  • remove from k0sctl-handler all other ssh-related coding

Formatting not enforced

Currently we've got many warnings for the README.md file generated by code scanning. This indicates a lack of configuration.

  • create an .editorconfig
  • create an .prettierrc
  • enable prettier option to use the editorconfig
  • enable prettier to proseWrap markdown on print width
  • use no-commit-to-branch in the pre-commit-config

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.