sklearn f1_score method has 'binary' as average default about flaml

[gianpd]

Hi @qingyun-wu, yes, you right, it is not very common to use f1 score for multiclass tasks. However, sometimes looking to the f1-micro / f1-macro could be required. By very good, I mean that if the dataset is unbalanced a micro f1 should be used more tough than a macro f1, however it could be an unnecessary complication.
Instead, I think the solution proposed by @sonichi should be great. If the task is multiclass f1 shouldn't be a valid option, while micro_f1/macro_f1 yes.

[sonichi]

or maybe we should ask users to specify 'micro_f1', 'macro_f1', etc. as metrics.

[sonichi]

@qingyun-wu do you agree? If so, I'd encourage @gianpdomiziani to create a PR. Thanks.

[qingyun-wu]

I agree. Thanks!

[qingyun-wu]

Hi @gianpdomiziani,

Thanks for reporting this issue. It seems f1 score is not a commonly used metric for multiclass classification tasks. So it is not clear to me how we should properly chose the 'best' metric in this case.

Among the several average options ['micro', 'macro', 'weighted', 'samples'], 'micro', 'macro', 'weighted' all work well. Do you think it is reasonable to just set the average option to be one of these three in the case of the multiclass classification task?

Thank you!