Kube-state-metrics 20x spikes in memory usage at restart about kube-state-metrics

[dgrisonnet]

/triage accepted
/assign @rexagod

[mindw]

@zhoujoetan try excluding configmaps and secrets from the list of exported resources (--resources= command line option).
At least for me it dropped initial memory usage from ~400Mib to 24Mib.

Both CLI and helm chart have them included by default.

In my case Helm charts (which store the manifests in secrets by defaults) history were the main culprit for the I could not confirm is pagination is used which may mitigate this issue.

Hope this helps.

[rexagod]

kube-state-metrics version: v2.3.0

@zhoujoetan It seems you're on an outdated version that's no longer supported. Could you switch to one of the supported versions (preferably the latest release) and verify this issue still persists for you?

[zhoujoetan]

I have figured out the issue. We had a ton of configmaps objects that KSM read during startup time. Trimming those objects brought the memory usage back down. I am closing the issue now.

[nalshamaajc]

@zhoujoetan When you sam Configmap objects, you mean this cluste-rwide or was it something specific? Also when you say trimming was it like deleting unwanted CMs, or removing data from these CMs?