kube-apiserver is too spammy and I can not exclude it about fluentd-kubernetes-sumologic

[frankreno]

@mfornasa :There are multiple ways to filter the data. if you want to do it at the source.kubernetes.conf, you would need to use the fluentd tail plugin option, exclude_path, which I believe does not accept regex. You can also use some environment variables to specify a regex pattern to exclude by host, pod, namespace or container. This is documented in the README in the options section. Does this not meet your requirement?

[mfornasa]

@frankreno my understanding is that all the EXCLUDE options operate only on containers that are collected trough source.containers.conf.

The problem here is with a Kubernetes component called kube-apiserver which is explicitly collected trough source.kubernetes.conf (in this line). So, I don't see a way to exclude logs from kube-apiserver using EXCLUDE directives.

This is a generic problem, as kube-apiserver is installed part of Kubernbetes.

[frankreno]

@mfornasa : My apologies, I see now. So we need to expose the environment variables in the source.kubernetes.conf and that should then give you this capability. I will look at this as soon as I can.

[mfornasa]

Any news on this?

[frankreno]

@mfornasa : I have not had any cycles to work on this and may not for a couple of weeks. However, I am open to a PR. We should be be able to use a similar flow to that was done for the source.containers.conf in most cases. Some may require some more work.

[frankreno]

@mfornasa : i will update when this is in our docker hub.

[frankreno]

@mfornasa: The image with the fix for this is now in docker hub. Thanks for your help @flah00!

[mfornasa]

Tried:

   - name: EXCLUDE_PATH
          value: "/mnt/log/kube-apiserver.log"

Seems good. Thanks.