guyuchao / inas Goto Github PK

Hello big brother Gu,it's been a while. I have read some old paper these days. And i find that in the 《Universally slimmable Networks 》----(seems to be the first paper present the sandwich rule and inplace distillation，I thought it was bignas,but it wasn't).It detach the teacher_pred for further distillation since it seems to be able to save a lot of memory in GPU. And you emphasized not detach in oneshot_model.py. I wonder why. And,Thank you for your work, I have learned a lot.

Hello again! I have noticed that you provided your pretrained checkpoints checkpoint-28b11d7f.pth. If i modify the search space,how can i get my checkpoints, and how can i pretrain it? Is the pretrain process including pretraining the supernet backbone on imagenet? Thank you!

torch.nn.modules.module.ModuleAttributeError: 'iNASSupernet' object has no attribute 'module'.
This error when one epoch is trained and try to valid the supernet。I wonder if this is a bug? i use pytorch 1.7.1,seems to qualify.I am really looking forward for your help. And great work it is.

你好,

Traceback (most recent call last):
File "iNAS/train.py", line 246, in
train_pipeline(root_path)
File "iNAS/train.py", line 145, in train_pipeline
model = build_model(opt)
File "/output/iNAS/iNAS/models/init.py", line 27, in build_model
model = MODEL_REGISTRY.get(opt['model_type'])(opt)
File "/output/iNAS/iNAS/models/oneshot_model.py", line 40, in init
self.load_network(self.supernet, load_path, self.opt['path'].get('strict_load', True), param_key)
File "/output/iNAS/iNAS/models/base_model.py", line 245, in load_network
load_net = load_net[param_key]
KeyError: 'state_dict'
首先这两个文件权重一样吗, 其次 为什么会报这个错呢 按照你的log 文件来弄得

I have tried different search space, and i find that sometimes i can't search the kernel in backbone stage1 and stage2,(torch.nn.modules.module.ModuleAttributeError: 'SMSU' object has no attribute 'ks_5' )and the deeper stage kernel in backbone is searchable. And sometime i can seach the kernel in stage 1,2, I 'm running my first version and it support the kernel seach in backbone stage 1 and 2. I don't know if i express myself clearly,so i say it in Chinese again.
就是我发现搜索空间的cfg里backbone stage1,2有的时候我可以设置kernel 搜索[3,5,7,9],有的时候我只能设置[3]它才不会报错， (torch.nn.modules.module.ModuleAttributeError: 'SMSU' object has no attribute 'ks_5' )，我的第一版搜索只搜索了骨干，之后每个stage的kernel我都可以自由的搜索，第二版我加入了transport的搜索空间，然后导致了我backbone的stage1和2的kernel不能搜了别的都没问题，请问是什么原因呢。

Hello! I've transfer your great work to other task,and I find the bn_finetuning process is super time consuming ,especially during the searching process(one 3080Ti), since each candidate need to get bn finetune(i set batchsize 32,iter 100) before validation. Is this process really nessesary? Is it beacause the weight comes from the supernet so the parameter in BN is chaotic makes it needs to finetune? I 'm a rookie in NAS, thank you!

Excellent work!
I wanna cite this paper in my work.
could you provide the parameter size and flops of the model which is searching on the GPU?