Not able to fetch snapscheduler metrics about snapscheduler

[JohnStrunk]

I can confirm the issue, at least w/ the Helm chart. Is that how you deployed?

[prasanjit-enginprogam]

@JohnStrunk : Yes i used Helm to deploy.

[prasanjit-enginprogam]

@JohnStrunk : Let me know what version/tag should I use after your fix.

[JohnStrunk]

Ok. I'll put this into a release in the next couple days.

[JohnStrunk]

A new release has been published: https://artifacthub.io/packages/helm/backube-helm-charts/snapscheduler
The metrics port should be accessible.

[prasanjit-enginprogam]

@JohnStrunk : are you going to add a new tag in https://quay.io/repository/cloudops/snapscheduler?tab=tags .. currently i can only see 1.1.1

[JohnStrunk]

cloudops isn't an official source. I don't know what those images are.
The snapscheduler container repo is in the backube org on quay:

$ skopeo list-tags docker://quay.io/backube/snapscheduler
{
    "Repository": "quay.io/backube/snapscheduler",
    "Tags": [
        "1.0.0",
        "1.1.0",
        "1.1.1",
        "latest",
        "1.2.0"
    ]
}

v1.2.0 is from today, as is the helm chart v1.3.0 on artifacthub.

[prasanjit-enginprogam]

Great thanks

[prasanjit-enginprogam]

@JohnStrunk: i was thinking if there is a way to add backup status in the metrics endpoint.. so that we can hook this up to grafana dashboard and hook up pagerduty(oncall) alerts based on it. what are your thoughts?

if you feel this makes sense and is something which is doable, I can create a new feature request for the same.

[JohnStrunk]

If there are particular metrics your looking for, please open an issue describing them (or a discussion thread).

Right now, snapscheduler doesn't monitor the snapshots that it creates. It just creates the VolumeSnapshot object and walks away. I'm guessing you'd want something to ensure it becomes readyToUse within some timeframe, but I have no idea how to bound that (AWS can take a looong time for big volumes).

Simple metrics about how many snapshots were created are much easier, but I'm not sure how useful that is.

[prasanjit-enginprogam]

@JohnStrunk: yes, correct, have some sort of a watcher pod that keeps track of backups happening and checks the readyToUse flag and exposes it as metrics from the endpoint which then can be scrapped by Prometheus or other observability tools.