RuntimeError of test_speedup.py about llm-pruner

[horseee]

Hi. The test_speedup.py only supports the testing on GPU currently. Have you tested this script on GPU?

[horseee]

Hi. We update the test_speedup.py to support the testing on CPU.
Please let me know if the bug still exists with the current version.

[sbwww]

Are there official FLOPs statistics on LLaMa-7B and its pruned version?

[horseee]

The FLOPs of the base model and the pruned model are:

[sbwww]

The FLOPs of the base model and the pruned model are:

here #MACs is equivalent to #FLOPs right? I just thought that they could be two different things.

[horseee]

The FLOPs of the base model and the pruned model are:

here #MACs is equivalent to #FLOPs right? I just thought that they could be two different things.

The results presented in the table is MACs rather than FLOPs and we didn't calculated the exact FLOPs of the model (roughly 2 times of MACs). To prevent any potential confusion, we chose not to use the term FLOPs in our paper, and thus all the results are MACs and not equivalent to FLOPs.

[Daisy5296]

Hi. The test_speedup.py only supports the testing on GPU currently. Have you tested this script on GPU?

Hi thanks for reply! Actually I was using GPU to run this code before.

Now I've tried the new version, when ran on CPU it worked fine. But if ran on GPU, the same error still existed.

[horseee]

Hi. The test_speedup.py only supports the testing on GPU currently. Have you tested this script on GPU?

Hi thanks for reply! Actually I was using GPU to run this code before.

Now I've tried the new version, when ran on CPU it worked fine. But if ran on GPU, the same error still existed.

Hi. I updated the code and fixed a bug that might cause this issue. You can give it another try, and if it still doesn't work, could you please provide your runtime environment?

[Daisy5296]

Thanks for rapid reply! I've also made the same corrections following your code for the CPU implementations, and it works now. Thanks again!