why not quantize the activation of the last conv layer in a block about brecq

[yhhhli]

Hi thanks for your comment,

Indeed there is some disagreement on how to insert activation quantization node. As you point out, some hardware can use this computation graph (like TensorRT) and some others cannot. However, many works also choose to quantize the input and the weights of a Conv2d layer and do not deal with a shortcut layer. In this case, we believe our method will have lower accuracy. It is not possible to find the same setting for all methods we compared.

[frankgt]

Hi
Thanks for your reply.
Indeed, the quantization process depends on the implementation of the NPU hardware.
Another questionm, I am using single 2080Ti GPU to run your code, CUDA out of memory error occurs for the deep networks and large bacth size(64). I have to use small batch size, 32 for mobilenetV2 or even 16 for resnet50.
Do you have the same problem? Any suggestion for that?

[yhhhli]

Good question,

in fact the GPU memory is mainly cost by storing the input & output for block. So there are 2 ways to deal with that:

1. Use multi-GPU reconstruction, we release the code of this using torch.distributed.
2. Reduce the size of calibration set, for example 512 or 768