liuxingbin / dbot Goto Github PK

Hi Authors,

Thanks for your work.

Could we have the code for the property analysis described in the paper as well (i.e., averaged attention distance and SVD).

Hi, Authors，
Thank you for your good work!
I want to know when you distill from the data-richer teacher, what is the data you use, IN1K or IN1K+400M ITp?
I see in your table 8, you say you use IN1K+400M ITp, so I wonder if the good performance is due to the rich data, not the method. I suppose here you want to show the data of teacher, not the student, is that true?
Thanks!

Hi, I notice that the settings for object detection are not consistant in your paper and in this repo.
For ViT-Base model, the setting in your paper is {cascade-maskrcnn; epoch:1x; layerdecay: 0.75}, while the setting in this repo is {cascade-maskrcnn; epoch:3x; layerdecay: 0.65}. So I want to know which setting should we use to reproduce your work.

I would like to ask about k-NN and Linear Probing. In your paper, you evaluated dBOT for fine-tuning, but what about k-NN and Linear Probing? If you have evaluated k-NN and Linear Probing in dBOT, it would be great to know how well they are doing.

hi, i am confused about the random teacher 0th performance. intuitively speaking, a model with random weights will not exhibit a reasonable performance. so i want to know whether there are some technology applied to this teacher.

Hi, Authors,
Thank you for good work!
I am confused about the paper:

1. you say the teacher is not important, but the best results is done by using data-richer teacher, exceed others by large margin.
2. you say the best practice is multi-stage by distill itself, but the best results is done by 1 stage with longer epochs and good teacher.
   So I am confused. Is there some point I don't understand?

I am new to self-supervised learning, what is the minimum hardware required to use dBOT? Is it possible to learn with my own dataset on a single GPU?