aamirraihan / swat Goto Github PK

Hi!

Thanks for your cool work. The results are so awesome.
Could you please tell me where is the code of 'structured pruning activations during backward'? Thank you very much!

I am trying to install SWAT package using pytorch 1.3.1, pytorch 1.4 (and latest versions) but facing the following error.

hello-extension.cpp:174:10: error: could not convert ‘out’ from ‘std::tuple<at::Tensor, at::Tensor, at::Tensor, at::Tensor>’ to ‘std::tuple<at::Tensor, at::Tensor, at::Tensor>’
return out;
^~~

It seems like it is a pytorch version error. Please let me know how to install it properly.
Thanks

I want to use SWAT in my FL frame using SWAT's sparsity training，what should I do?

I pulled the docker you provided，how to use it？Do I need run the mypkg？

Hi Md Aamir Raihan,

Thank you so much for the open-source code and great work! I have a question here, I can run your code, and also I believe this work does theoretically speed up training, but when I do a speed comparison between a regular resnet18 and the resnet18 in your script, it doesn't actually speed up. I think because you have customized some layers in your resnet18.

So my question is, how do you measure the decrease in flops or how do you measure the decrease in time?

Thanks again for the code and paper!

Hi, thanks for the great work!
I know little about sparse computation, and I'm looking for methods that can speed up training process on common devices (e.g. NVIDIA GPU). I'm just wondering, if I adopt the SWAT method on common GPU devices (no special accelerator for sparse computation), will there be any benefits in terms of speed up?