mattermost / rotatorctl

CLI tool for smoothing and accelerating k8s cluster upgrades and node rotations.

rotatorctl (Rotator CLI)

rotatorctl (Rotator CLI) uses the Mattermost Rotator module to smooth and accelerate k8s cluster upgrades and node rotations. It offers automation on an autoscaling group recognition and flexibility on options such as how fast to rotate nodes, drain retries, waiting time between rotations and drains as well as mater/worker node separation.

How to use

Get or build the release binaries and use it as a CLI locally or in any pipeline to automate nodes rotation.

Building

Simply run the following:

go install ./cmd/rotatorctl
alias rotatorctl='$HOME/$GOROOT/bin/rotatorctl'

or use the make shorthands

For Linux binary: make build

For MacOS (darwin) binary: make build-mac

Running

To rotate a cluster:

rotatorctl rotate --cluster <cluster_name>  --wait-between-rotations 30 --wait-between-drains 60 --max-scaling 4 --evict-grace-period 30

Other Setup

For the rotatorctl to run access to both the AWS account and the K8s cluster, it is required to be able to do actions such as, DescribeInstances, DetachInstances, TerminateInstances, DescribeAutoScalingGroups, as well as drain, kill, evict pods, etc.

The relevant AWS Access and Secret key pair should be exported and k8s access should be provided via a passed clientset or a locally exported k8s config.

rotatorctl's Issues

Is this tool useful for EKS with Managed Node Group?

Scenario

Consider the K8s cluster is EKS with Managed Node Group. In case of EKS upgrade, we need to rotate out the Managed Node Groups with the new AMI with updated version of Kubelet and other changes. AWS EKS do the rotation painlessly, however, just trying to implement some customisation to manage timing. Saw the parameter "--wait-between-rotations" in rotatorctl. Just trying to explore more here.

Questions

1. Does this support EKS with Managed Node Group by any chance?
2. Any similar tool available for EKS with Managed Node Group?

I went through the doc https://mattermost.com/blog/automate-eks-node-rotation/ and it says following:

After we adopted AWS EKS managed nodes, we simplified our Gitlab pipelines by dropping our custom rotation mechanism, as managed nodes offer natively their own.

Just confirming.

Thanks in advance!