sparsity about dynamic-network-surgery-caffe-reimplementation

[HolmesShuan]

To be honest, I haven't tried 90% sparse convolution layer. According to my knowledge, that kind of sparsity could only exist in fully connected layers or a few convolution layers without a huge loss in accuracy. I have tried 50% ConV on resnet. About 2% higher than baseline @top-5 on ImageNet.

I have same result with you, did you have other nice result about pruning with other papers? I tried this work and deep compression and ssl,but I find these sparsity methods cost a lot time and without a good acc and sparsity

[HolmesShuan]

DNS seems the fastest algorithm,so far. You want a light network or just to prune the pre-trained dense network?

I foucs on model compression, I think light work is more important for Embedded system such as phone, but a good network is time-consuming to find, I think pruning + low bit is a good choice,do you have other advices for model compression, especially for fast inference? or will a chat with wechat?

[HolmesShuan]

U may interest in BMXNET , SqueezeNet and PerfBLAS. There are other high-performance Binary GEMM libs, u may turn to GitHub Search for more help.