sunshangquan / logit-standardization-kd Goto Github PK
View Code? Open in Web Editor NEW[CVPR 2024 Highlight] Logit Standardization in Knowledge Distillation
logit-standardization-kd's People
Contributors
Stargazers
Watchers
logit-standardization-kd's Issues
想问一下logit具体的计算方式
作者您好，以cifar100为例，想问一下您具体是怎么计算每一类的logit的，是将所有的测试集输入到模型中，然后将所有输出的logit求和，最终除以测试集大小吗。
Distributed training for ImageNet
Hi, may I know if your ImageNet experiments ran on multiple GPUS, and if multiple-GPU training is supported? Thanks.
Missing configuration to reproduce "KD + Ours" result of ResNet-18 student with ResNet-34 teacher
Hi @sunshangquan ,
Congratulations for the accepted paper!
I wanted to reproduce the ImageNet result for ResNet-18 student with ResNet-34 teacher
README.md says
python tools/train.py --cfg configs/imagenet/r34_r18/kd_ours.yamlbut there are no file names ending with _ours.yaml in configs/imagenet/r34_r18/
Could you share the configuration file to reproduce the result?
迁移实验 & 学生模型
- 请问是否考虑过将蒸馏后的模型从CIFAR-100迁移到STL-10数据上?
- 请问是否可以提供学生模型?
交叉熵损失的作用是什么
- 您在算法部分出现Lce，对于它的作用我在文章中并没有找寻到（可能是我没有找到，如果是这样非常抱歉）
- 我现在在进行互相知识蒸馏，没有使用Lce，仅仅只使用了预处理部分代码可是效果并不理想
希望您抽空解答一下我的疑问，非常感谢
实现层面
作者你好，请问在实现层面是否仅有对学生和老师的logit做Normalize操作呢？是否还有其它我没发现的地方需要处理？正在学习您的工作，希望可以得到回复。
关于蒸馏Vit的问题
请问蒸馏vit的代码是各个蒸馏方法通用的吗？另外，请问vit可以作为教师模型吗
想问一下关于CE_WEIGHT的问题
您好，您的工作非常棒！我最近正在学习知识蒸馏这块的最近工作，我看到您关于LOSS.CE_WEIGHT的设置在KD上是0.1，而在其他的蒸馏方法上均是1.0，想问一下这是因为什么呢，非常感谢 : )
config
Dear author：
能提供下在cifar100上蒸馏ResNet50，学生模型为WRN-16-2的配置文件吗，想学习一下
怎么迁移到多标签分类任务上？
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.