policy-reporter presents incomplete results about policy-reporter

[mikebryant]

Sigh, and I realise having written this we're actually not on the latest helm chart, so I'll go and upgrade that and see if that happens to fix things

[mikebryant]

Upgraded to latest chart (2.24.0), which gives image version ghcr.io/kyverno/policy-reporter:2.20.0 - same issue

[fjogeleit]

So you mean its not showing all namespaces at the beginning but after some time the list is complete? Or the list is never fully complete?

In general means ready that the listener starts working, all information including namespaces comes from PolicyReports and PolicyReporter needs to process all existing PolicyReports before all data is available.

I never have an indicator when all at start time existing reports are processed because they come one by one from the listener. Also depending on cluster, amount of policies and resources, the time to process everything is very different.

[mikebryant]

After some time the list is complete.

It's being particularly noticeable right now on our sbx environment, because it only has one replica and it keeps being restarted by the VPA controller when the resource estimate changes, and each time the UI stops being accurate for 5-10 minutes.

The issue I have is I want to be able to point our users at this as a tool to look at the state of their apps - but it's tricky when the data may be inaccurate, but presented as if it's accurate. So for my use-case, I'd prefer it not be available until it's ready, rather than responding with missing data.

I've seen other projects have similar requirements, e.g. kubernetes/kubernetes#113763 & https://github.com/istio/istio/blob/master/pkg/kube/kclient/client.go#L190-L203

[fjogeleit]

So you would prefer that policy reporter needs 5 to 10 minutes to be marked as running?

I am not sure if your example hits this situation. HasSynced is already used as indicator for readiness:

https://github.com/kyverno/policy-reporter/blob/main/pkg/api/handler.go#L32

But this dosn't mean that everything is processed. The API is based on an internal Database, which will be filled after the sync by processing each PR one by one. The API does not get its data directly from the K8s API. And this process takes time depending on the amount of reports and results.

I have no way to know or to check if all initial reports are processed at least once.

[mikebryant]

So you would prefer that policy reporter needs 5 to 10 minutes to be marked as running?

Tbh, yes - I'd like the option for that at least. My perspective is unavailable and inaccurate are both bad. (Though I suppose available but with a warning saying it's incomplete would also work)

I am not sure if your example hits this situation. HasSynced is already used as indicator for readiness:

🤔 Hmm, I know that things like istio do this somehow (e.g. you don't want to start allowing traffic before you know you've processed all AuthorizationPolicy)

I'm happy to take a look at trying to achieve this.

I guess my question is - is this something you'd be interested in accepting as a PR? (Possibly as an option?)

[fjogeleit]

As an Option sure, maybe one way could be to use an external DB, which is already possible and improve the behavior after a restart. So it keeps already existing data, only updates changes and remove no longer existing information (which is the hardest part).

I need to focus on the UI for now, so if you would like to take a look, I am happy about each contribution.