Hi，where is the KL loss in your code about wsi-finetuning

[jeremy-aitrics]

Thank you for your interesting work!

I have a same question with weliaicunzai.
I can't find the KL divergence loss in your code. Where did you implement the KL divergence loss?

Thanks.

[invoker-LL]

MSE and KL divergence are both metrics/loss for distribution alignment as used in Knowledge Distillation. Since Bernoulli distribution is easy with given constant probability like p=0.2, it can be seen as a soft-target for a binary classification, thus we can also use CrossEntropy to replace the KL given pre-defined distribution. I will provide this code soon.