preresnet layer inconsistent when apply custom config about rethinking-network-pruning

[Eric-mingjie]

I strongly recommend not using cfg in this way. (cfg should be automatically generated when running the pruning code.) The reason for your problem is that you didn't set the channel_selection layer accordingly.

Specifically, you need to choose four channels in channel_selection layer to form as the output.

[Eric-mingjie]

The first convolution is not represented in cfg. The first 16 in cfg refers to the channels in the first convolution of the first residual block.

[huangbiubiu]

But the output channel of the first convolution is fixed to 16. When the first channel in cfg is pruned, there will be an error.

Reproduce the error (assume the first convolution of the first residual block was pruned to 4):

import torch

from models import resnet


def main():
    # follow the depth in the paper
    depth = 164
    n = (depth - 2) // 9
    # assume the first layer is pruned to 4 (default 16)
    # cfg = [[16, 16, 16], [64, 16, 16] * (n - 1), [64, 32, 32], [128, 32, 32] * (n - 1), [128, 64, 64],
    #        [256, 64, 64] * (n - 1), [256]]
    cfg = [[4, 16, 16], [64, 16, 16] * (n - 1), [64, 32, 32], [128, 32, 32] * (n - 1), [128, 64, 64],
           [256, 64, 64] * (n - 1), [256]]
    cfg = [item for sub_list in cfg for item in sub_list]

    newmodel = resnet(depth=164, dataset="cifar100", cfg=cfg)
    input_demo = torch.zeros(4, 3, 32, 32)
    output = newmodel(input_demo)

    pass


if __name__ == '__main__':
    main()

To run the code, paste the code to cifar/network-slimming/reproduce.py and run it.

The error message:

Traceback (most recent call last):
  File "/home/*****/projects/rethinking-network-pruning/cifar/network-slimming/reproduce.py", line 23, in <module>
    main()
  File "/home/*****/projects/rethinking-network-pruning/cifar/network-slimming/reproduce.py", line 17, in main
    output = newmodel(input_demo)
  File "/home/*****/anaconda3/envs/pruning/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/*****/projects/rethinking-network-pruning/cifar/network-slimming/models/preresnet.py", line 112, in forward
    x = self.layer1(x)  # 32x32
  File "/home/*****/anaconda3/envs/pruning/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/*****/anaconda3/envs/pruning/lib/python3.6/site-packages/torch/nn/modules/container.py", line 92, in forward
    input = module(input)
  File "/home/*****/anaconda3/envs/pruning/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/*****/projects/rethinking-network-pruning/cifar/network-slimming/models/preresnet.py", line 38, in forward
    out = self.conv1(out)
  File "/home/*****/anaconda3/envs/pruning/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/*****/anaconda3/envs/pruning/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 338, in forward
    self.padding, self.dilation, self.groups)
RuntimeError: Given groups=1, weight of size 16 4 1 1, expected input[4, 16, 224, 224] to have 4 channels, but got 16 channels instead

Process finished with exit code 1