eclipsess / chip_neurips2021 Goto Github PK

hi，您好，请问代码中的预训练模型具体是由什么代码训练出来的呢，有链接吗，万分感谢

First of all, I appreciate your great work. 

As far as I know, generally, resnet_56 for the CIFAR10 dataset is composed of bottleneck blocks.
However, the resnet_56 model in this code is composed of basic blocks. And HRank, too.

Is there any reason to build the resnet_56 model like this?

您好，请问resnet56在cifar10上裁剪大约43%的参数后，是多次结果取最高值吗。在您最新给出的微调参数下我运行3次后得到的结果分别是93.99，93.64，93.78，没有达到论文汇总94.16的效果。这是实验日志https://drive.google.com/drive/folders/1v4UNqc7AwDNy-y36U2KOtbHYPO8c_P2w?usp=sharing。当使用论文中给出的微调参数，lr=0.01、epoch=300、weight decay=0.005、momentum=0.9、lr type=cos、label smooth=0.0，运行四次的结果分别是93.9，93.97，94.01，93.94。请问是参数设置是有误差，还是实验结果要多运行几次去最高值呢？非常感谢。

作者您好：

首先非常感谢您能分享这么棒的工作。

我在运行calculate_feature_maps.py的时候遇到cifar10的train_loader不能进行有效遍历（多余的参数位置，即不能for batch_idx, (inputs, targets) in enumerate(train_loader):），这个bugs我用了自己的cifar train_loader以修正。

我在运行calculate_ci.py的时候遇到以下错误：
File "calculate_ci.py", line 22, in ci_score
conv_output = np.round(np.load(path_conv), 4)
File "/home/warwick/anaconda3/lib/python3.7/site-packages/numpy/lib/npyio.py", line 428, in load
fid = open(os_fspath(file), "rb")
FileNotFoundError: [Errno 2] No such file or directory: './feature_conv/resnet_56_repeat5/feature_convtensor(1).npy'

原因是找不到59行的path_conv = "./feature_conv/resnet_56_repeat5/feature_convtensor({0}).npy".format(str(index))
而且第60行的path_nuc = "./feature_conv_nuc/resnet_56_repeat5/feature_conv_nuctensor({0}).npy".format(str(index))
以及parser.add_argument('--feature_map_dir',type=str,default='./conv_feature_map',help='feature maps dir')都没有用到。

请问代码是不是漏掉了中间环节。
期待您的回复
谢谢

Hi,Thanks for your wonderful work. However, in ResNet56, I could not get the same result in 300 epochs when the flops was set to 47.4%.

Hi, thank you for sharing the excellent method!
I noticed python calculate_ci.py is running on CPU, not GPU.
It took me very little time to run this program for cifar but a long time for imagenet.
How long does it take to run python calculate_ci.py for imagenet?

谢谢您的回答。

你好. 非常感谢你的工作. 有几个问题请教一下:

1. 如果是手动指定这个ratio, 那么在指定过程中有什么insight可以分享吗?
2. 请问有尝试过自动决定每层的ratio吗?

Hi,

thanks for the great work! I have a question about how to acquire Figure 4 in the original paper. Could you share how you demo for Figure 4?

Thanks

原论文中resnet56在cifar10上的微调的weight decay 为0.05，请问是不是笔误了呢，实际上应该是0.005.

Hello, how do you calculate your Params and Flops?

Here is the first few layer of the ResNet 50 in this implementation:
ResNet50(
(conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
0(maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
(layer1): ModuleList(
(0): Bottleneck(
(conv1): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
1(relu1): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
2(relu2): ReLU(inplace=True)
(conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
3(relu3): ReLU(inplace=True)
(downsample): Sequential(
(0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)

And here is the hook code from calculate_feature_maps.py:

elif args.arch == 'resnet_50':
    cov_layer = eval('model.maxpool')
    handler = cov_layer.register_forward_hook(get_feature_hook)
    inference()
    handler.remove()
    # ResNet50 per bottleneck
    for i in range(4):
        block = eval('model.layer%d' % (i + 1))
        for j in range(model.num_blocks[i]):
            cov_layer = block[j].relu1
            handler = cov_layer.register_forward_hook(get_feature_hook)
            inference()
            handler.remove()

            cov_layer = block[j].relu2
            handler = cov_layer.register_forward_hook(get_feature_hook)
            inference()
            handler.remove()

            cov_layer = block[j].relu3
            handler = cov_layer.register_forward_hook(get_feature_hook)
            inference()
            handler.remove()

            if j == 0:
                cov_layer = block[j].relu3
                handler = cov_layer.register_forward_hook(get_feature_hook)
                inference()
                handler.remove()

In each Bottleneck block, there is only one relu3, and in all first Bottleneck block of every layer, which specified by if j == 0:
I suppose the special case j == 0 means the corresponding special operation about the downsampleing layer.
Did I miss something or misunderstand something?

First, thank you for contribution on structure pruning.
I am impressed by this neat and efficient structure pruning method. However, I'd like to know how to decide sparsity on each layer for any model since I did not find any part mentioning this on paper and sparsity rate could make huge impact no matter on inference speed or accuracy.

Thank you.

Thanks for your work. I am confused about the desired number of filters to be preserved k in Algorithm 1. If k is predesigned, why not prun the model directly. Is there any advantages to pruning the model with more computation about nuclear norm?

你好，我使用您的代码cal_flops_params.py测试VGG-16未剪枝前的网络计算量和参数量，得到的结果：314.572M(FLOPs)、14.992M（Params），但是您在论文的结果是：313.73MM(FLOPs)、14.98M（Params），想请教下关于结果差距的原因？

您好，请问您在MobileNet之类的新型网络中测试过该方法吗？如MobileNetV2，效果如何呢？

First, I appreciate your insightful work.
But when runing the code python calculate_ci.py --arch resnet_56 --repeat 5 --num_layers 55, I encounter the following error:

Traceback (most recent call last):
File "/CHIP_NeurIPS2021/calculate_ci.py", line 91, in
main()
File "/CHIP_NeurIPS2021/calculate_ci.py", line 81, in main
ci = mean_repeat_ci(repeat, num_layers)
File "/CHIP_NeurIPS2021/calculate_ci.py", line 75, in mean_repeat_ci
return np.array(layer_ci_mean_total)
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (55,) + in homogeneous part.

I followed the instructions in the README.md. Could you please help me to address this error?