checkly / checkly-operator

Kubernetes checkly operator

checkly-operator

A kubernetes operator for checklyhq.com.

The operator can create checklyhq.com checks, groups and alert channels based of kubernetes CRDs and Ingress object annotations.

Documentation

Please see our docs for more details on how to install and use the operator.

Development

direnv

We're using direnv to manage environment variables for this project (or export them manually and you can skip this step). Make sure you generate a checkly API key and you get the account ID as well.

touch .envrc
echo "export CHECKLY_API_KEY=foorbarbaz" > .envrc
echo "export CHECKLY_ACCOUNT_ID=randomnumbers" >> .envrc
direnv allow .

Makefile

Make sure your current kubectl context is set to the appropriate kubernetes cluster where you want to test the operator, then run

kubectl apply -f config/crd/bases/k8s.checklyhq.com_apichecks.yaml
kubectl apply -f config/crd/bases/k8s.checklyhq.com_groups.yaml
kubectl apply -f config/crd/bases/k8s.checklyhq.com_alertchannels.yaml
make run

If you update any of the types for the CRD, run

make manifests

and re-apply the CRD.

Testing the controller

Unit and integration tests

• Make sure your kubectl context is set to your local k8s cluster
• Run USE_EXISTING_CLUSTER=true make test
• To see coverage run go tool cover -html=cover.out

Running locally

See docs for details.

Source material

Sources used for kick starting this project:

• https://sdk.operatorframework.io/docs/building-operators/golang/tutorial/
• https://kubernetes.io/blog/2021/06/21/writing-a-controller-for-pod-labels/
• https://github.com/checkly/checkly-go-sdk
• https://docs.okd.io/latest/operators/operator_sdk/golang/osdk-golang-tutorial.html
• https://sdk.operatorframework.io/docs/building-operators/golang/advanced-topics/#external-resources
• https://book.kubebuilder.io/cronjob-tutorial/writing-tests.html

Versions

We're using the following versions of packages:

• operator-sdk 1.31.0
• golang 1.21

checkly-operator's Issues

Automatic tags from labels causes errors in prometheus integration

Currently we're taking all the labels from the existing ApiCheck or Group resources, any automatic labels from other services will be automatically added as tags, this works as intended.

Checklyhq.com has a prometheus integration, this takes the tags on a check and makes it into a metric label.

At IMG Arena we had a restore procedure from a backup and this created additional labels on the ApiCheck resources, which created the tags on the checklyhq.com API Check as seen below:

The prometheus integration broke and we were unable to scrape the checklyhq prometheus endpoint as we received an error (do not have the error anymore unfortunately, will try to reproduce).

Removing the tags from the checklyhq.com api check fixed the problem.

Failed resource creation is not clear on status

As seen in the youtube video with Stefan, we found that creating an API check that actually fails to be created, it is never clear from a kubectl get <resource-type> <resource-name> that it failed.

Feature Request: Webhook Alert Channels

Hey,

we're experimenting with the Checkly Operator and hit a limitation.
We're using GitLab Alerts and would like to use the Webhook Alert Channel via this operator.
Currently only OpsGenie and E-Mail is supported.

Would be great if this could be added, for the moment we're implementing a workaround via Checkly CLI/API.

Groups have to be in the same namespace as apiChecks

We are tying each ApiCheck to a Group, unfortunately at the moment each group has to be inside the same namespace as the ApiCheck, we can not reference groups across namespaces.

In case you are using multiple namespaces for your applications and you have an ingress in all of them, you can not create one group for all, but you need to create a group / namespace.

I'm leaving this issue here to make sure this is documented and hopefully in the future a solution is found.

Checks created are not functioning

Any check created through the operator doesn't work on checklyhq.com. Reason is unclear.

The checks are created, they are valid, but they always fail. Same checks created manually succeed.

Checkly-operator version: v1.2.3