[ENHANCEMENT] Ability to use different metrics for calculating forecasting errors about scikit-hts

[vtoliveira]

Forget about it, sklearn already implements in this way, you choose if you want multioutput or not, I will try to follow their guidelines and practice so we have a robust metric. I will start with rmsse, the one used in M5 current competition.

[vtoliveira]

I will try to work on that, and give feedback as I advance. So for custom metrics you are also thinking
in adding a hts.metrics interface as well? Because I am aware that sklearn does not implement a lot of them, just basic ones.

So in my mind would be something like:

from hts.metrics import root_mean_squared_scaled_error

Also, we have this Hyndman's paper that discuss all metrics dealing with time series and the ones for intermittent data as well. (For what I have seen, almost all Hierarchical data follows some kind of intermittend demand)

[vtoliveira]

Another question that I was thinking of, considering the case implemented above, the metric should automatically calculated rmse for each node or should we leave it to the user?

Because we have two different cases:

• First, user passing metric as string / callable, in base.py each series is calculated at time.
• Second, user get model.predict and compare with a test dataset. And we would have here for each y_true, y_pred shape -> (n_samples, n_nodes)