Comments (5)
- 在retina和fcos这几个上面只用了 general distribution,没有使用dfl和qfl
- retina的bbox_coder实际上没用到
from ld.
- 在retina和fcos这几个上面只用了 general distribution,没有使用dfl和qfl
- retina的bbox_coder实际上没用到
感谢回复~!
可以理解为采用generalized distribution是loss_bbox=dict(type='GIoULoss')这个决定的吗?
另外,关于蒸馏实验,我之前采用您的方法做实验,确实对于同样使用ResNet+FPN的情况下,用ResNet101指导ResNet18有很大提高
但是这几天尝试了使用MobileNet作为student,用ResNet-101的结果作为teacher会有一些精度下降,不知道是不是蒸馏对网络结构有要求,即网络结构近似的效果会更好?
(但是我理解的蒸馏是将teacher网络的分类结果,这里是推演出的边框概率位置,作为指导学生的,所以感觉可能是没关系的?是不是应该往调整蒸馏学习的超参数方向试试呢?谢谢~!
from ld.
准确来说有三个超参值得考虑,一个是边界被离散化成概率分布的离散程度n (小区间的个数),gfocal是16,对于rotated RetinaNet我们发现8更好,因此n可能会有一定的影响,不过差别不会太大。
第二个是温度\tau,这影响着概率分布的软化程度。一般而言会在5~15这个范围比较好。
第三个是loss weight,这个可以调一调。
我建议是按照这个顺序逐一去确定。
from ld.
没做过mobilenet的实验,你可以用resnet18或者34做teacher试试
from ld.
准确来说有三个超参值得考虑,一个是边界被离散化成概率分布的离散程度n (小区间的个数),gfocal是16,对于rotated RetinaNet我们发现8更好,因此n可能会有一定的影响,不过差别不会太大。
第二个是温度\tau,这影响着概率分布的软化程度。一般而言会在5~15这个范围比较好。
第三个是loss weight,这个可以调一调。
我建议是按照这个顺序逐一去确定。
谢谢您的回复!我这几天重新做了实验,发现teacher选择上也有一定影响
之前用res18做student的话,LD之后效果最好的是res50
mobile我试了还是res101效果最好,猜测可能因为res101下AP^S最高,补足了mobile的AP^S低的问题
我再按您的方法试一试~感谢!
from ld.
Related Issues (20)
- I reimplement your model (Teacher: R101, Student: R50, Detection head: Foveabox) and got RuntimeError: shape '[-1, 68]' is invalid for input of size 115200 HOT 5
- Welcome update to OpenMMLab 2.0 HOT 2
- 请问下代码里的教师模型是跟着学生模型一起训练的吗 HOT 2
- If I set output_feature = False , Code don't work HOT 1
- 参考install.md 里面配置的环境无法找到cuda HOT 13
- 简易版本复现问题 HOT 5
- 将DKD与LD结合的问题 HOT 8
- Implement LD on VisDrone dataset HOT 5
- 学生网络与教师网络性能差距较大
- mmpycocotools HOT 1
- 请问下是否可以迁移到centernet呢 HOT 2
- 知识蒸馏 HOT 10
- 请问是使用 ldv2/ld_r50_gflv2_r101_fpn_1x.py 这个文件来复现 Temperature τ in LD 这个实验吗? HOT 1
- kd_loss implementation issue HOT 9
- 关于数据集minicoco HOT 4
- About Adaptive Layer HOT 2
- Can we pass different set of image to teacher and student. HOT 6
- samples_per_gpu HOT 1
- loadcheckpoint出现问题 HOT 2
- Knowledge distillation related
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ld.