richarizardd / self-supervised-vit-path Goto Github PK

Hi.

Since reading your paper, I'm quite confused about why did you use the loss function like above? or maybe the typos?
(I cannot find your train codes so only way to check is by making issues)

Loss function you used is some weird cause it measures the cross entropy of teacher distribution relative to student distribution and train student network.

Your reference paper DINO use different loss function as below, which measures the cross entropy of student distribution relative to teacher distribution and train student network.

Is there some missing points that I don't know or it's just typos?

Hi,
Thank you for your nice work.
I wander if you could tell me training time consumed and your GPU type.

from pl_bolts.models.self_supervised import resnets
from pl_bolts.utils.semi_supervised import Identity
The two libs can't be found in the project.

Thanks for your great work.
In the figure, there is a global pooling layer after transformer encoder in ViT backbone. In the original implementation of dino, only cls token is used for successive processing. I wonder if you use global pooling of all tokens instead of cls token in this step.

Hi @Richarizardd,

Super interesting work!
Would you mind providing me with the conditions/parameters you used for DINO? I am trying to replicate some parts of your paper, and would really appreciate if you could provide the code for DINO that you used. Furthermore, would you have any advice on the set of parameters I should employ when using a ResNet50 as a backbone encoder with DINO? Or have you only experimented with ViTs as the backbone?

Thanks in advance :)

Hi,

Could you share the code about training too?

Thanks

Thanks for your great work!
In the paper, you said you used 1038 TCGA-BRCA for training. The project TCGA-BRCA provides 1062 Diagnostic Slides and 1093 Tissue Slides, and I wonder which type you used.