Whitelisting or blacklisting metrics about hadoop-hdfs-fsimage-exporter

[marcelmay]

@zefir6 , you could try to reduce the buckets fsimage_user_fsize_bucket buckets for a start , or completely disable file size distribution for users.

Reduce the number of buckets:

# Configure file size distribution buckets, supporting IEC units of KiB, MiB, GiB, TiB, PiB
fileSizeDistributionBuckets: ['0','1MiB', '32MiB', '128MiB', '1GiB']

Disable any file size distribution on user level:

skipFileDistributionForUserStats : true

The file size distribution on user level is quite useful for catching culprits that create too many small files on HDFS though...

[zefir6]

Hey, thanks for reply :)
Regardless of how I set that option, I still get hundreds of lines like this:

fsimage_group_blocks{group_name="group.name",} 0.0
fsimage_user_blocks{user_name="user.name",} 0.0

(and the same for many other attributes)

According to my count, with every skip enabled, every user (even if he has one file), generates 15 lines in the exporter. I was just considering editing the code and trying to cut out the bit that generates user & group labels (as thats about peak of my java skills :) )

[marcelmay]

Can you provide me the output of number of metrics?

sort_desc(count({__name__=~"fsimage_.*"}) by (__name__))

Usually the mentioned before histogram metrics for file size distribution have a real impact (number of users X number of buckets ...).

[marcelmay]

... and the config if possible