About the consistency loss about randconv

[wildphoton]

Hi @yakexee, I think F.kl_div() defines inputs and targets in the way how NLL loss is used. Since NLL(input, target) = cross_entropy(target, input). I believe F.kl_div(input, target) = KL(target || input). You can check this with an easy example by comparing the results with a KL div function implemented by yourself. Since we are computing KL(p_aug || p_mixture), we called F.kl_div(p_mixture, p_aug). FYI, this loss is adapted from the Augmix's implementation. I hope it helps

[yakexee]

Thanks for the kind reply. That helps!