FreeMatch SAF loss is a negative value? about semi-supervised-learning

[Hhhhhhao]

Negative loss doesn't affect the gradient

[hzhz2020]

thank you for your fast reply!
I am just curious about the intuition behind this loss. By minimizing $L_{f}$ in eq11, aren't you pushing SumNorm (p˜t/h˜t) and SumNorm(p¯/h¯) to be further apart? I thought you want to make them closer?

[Hhhhhhao]

thank you for your fast reply! I am just curious about the intuition behind this loss. By minimizing Lf in eq11, aren't you pushing SumNorm (p˜t/h˜t) and SumNorm(p¯/h¯) to be further apart? I thought you want to make them closer?

The loss encourages fairness on average predictions. We expect the average predictions to be close to uniform. Reflected on the entropy loss, it corresponds to maximizing the entropy. We replace the target term with a momentum-smoothed average prediction for stability. That's the intuition behind this loss. I think the paper and relevant works mentioned in the paper discussed this in more detail.

[hzhz2020]

thank you for explaining it!