sklearn metrics error about blog_references

[magesh-technovator]

Can you verify that all your test labels and predictions labels are binary?

It should like the above one

And also please add a screenshot/traceback of the exception

[rajae-Bens]

Hi,

here is my labels format

and the predictions

and here is the error I got

Thanks for ur help

[magesh-technovator]

Reference from sklearn:
ValueError("Classification metrics can't handle a mix of {0} "
"and {1} targets".format(type_true, type_pred))

I think your prediction results are in multiclass-multioutput type, you can verify it here(https://github.com/scikit-learn/scikit-learn/blob/e6555decf8a72958d26d499deb17aabca41a562a/sklearn/utils/multiclass.py#L175)

And one more thing, Your model seems to output "0" for all classes those 5 samples, Can you please verify the predictions once?

And also check the step where you've transformed your output labels to fit for multi-label problem, something like this:

from sklearn.preprocessing import MultiLabelBinarizer
mlb = MultiLabelBinarizer()
test_labels = mlb.transform(test_categories)

[rajae-Bens]

Hi,

Thank s for ur answer. Actually, I solved the problem by modifying the format of my data, and then I used MultiLabelBinarizer to transform them into a convenient format. Now all the metrics are calculated correctly.
However, I have another question plz, I've read that when handling a multi-label classification problem, f1-score is more significant as an evaluation metric do u agree? also, I am getting a very good f1 micro score but the macro score is not that good with a difference of 3x% do u have an explanation for that?

thank s for ur help

[magesh-technovator]

Thanks for the response.

Regarding micro vs macro, it actually depends on your business needs,

Micro averaging takes into account the proportion for each class, but macro-averaging averages f1-score(or any metric) for all classes treating them equally.

If you have an imbalanced dataset(this is typical in real-world scenarios), micro-average is preferable.

And one more suggestion, if you have an imbalanced dataset, based on your business problem figure out the cost of mis-prediction and do some manual error analysis to get insights, and choose the model that gives better performance for your business need.

Example: Class A being mis-predicted as Class B will have a major impact, but if Class C gets mis-predicted as Class B it can be tolerated.