Unexpected behaviour for numerical fields above 100 about spark-privacy-preserver

[ThaminduR]

I fixed the function to do the intended task. Check the PR.
@mkffl @agessner

Test the function and add a comment here. If we are good to go, I'll merge the pr.

[ThaminduR]

Intended behavior was to group numerical values gracefully rather than taking the min and the max values of a partition. It seems like there is a logical error in the ``agg_numerical_column``` .

[mkffl]

Do you have any external reference that explains/shows what how this method should work, ideally with code that does not include any logical error?

It's a shame that agg_numerical_column does not work exactly as it should but I am happy to have a try at fixing it and submitting a PR. I just a need a pointer.

[agessner]

Any update on this?

[ThaminduR]

What the agg_numerical_column is intended to do is, take the min and max of the column and then round them up to the nearest x5 multiplier.

Intended behavior:

• A column with values ranging from 3 to 9 would be aggregated to 0-10
• A column with values ranging from 236 to12343 would be aggregated to 235-12345

If you have any suggestions to do this in another way, feel free to add those here.