Is there a theoretical range of values for the COMET regressor? about comet

[ricardorei]

Hi @alvations

Yes, the scores are unbounded for most COMET models (except wmt20-comet-da-qe that has a sigmoid in the end). Nonetheless, since they are trained to regress on a z-scores they usually range somewhere between -2.5 to 1.2. You can observe that in this plot:

That this plot from WMT20 en-de data which is a high resource and high quality language pair. You can expect this distribution to have a smaller mean for low resource languages.

The max value and min value should be more or less the same because the model does not "risk" much outside those values.

[alvations]

Thanks for the explanation! It'll make sense if we somehow distill the model and put a softmax at the final layer so that it spits out [0, 1.0].