vinsis / ternary-quantization Goto Github PK

Hey, do you try to train lots of epoches? I found that the accuracy will be reduced after 5 epoch, until to 10%. It really confused me, could u please give me a hand?

HI, @vinsis Thanks for the pytorch implementation of the TTN.
I have an issue about the code. In quantionization.py line14:

should full_precision_grad = a * full_precision_data * w_p_data + b * full_precision_data * w_n_data + c * full_precision_data * 1 be

full_precision_grad = a * grad_data* w_p_data + b * grad_data* w_n_data + c * grad_data* 1?

It seems the original gradient not used if employing full_precision_data .

Hello, I managed to perform your quantized neural network in my machine. however, the trained weights are like this (a kernel):
[[[ 0.0000, 0.0000, 0.4256],
[ 0.0000, 0.4256, -1.4813],
[ 0.0000, 0.0000, 0.0000]]],
all the weights I got is like this. It seems like not a ternary weights which should be [-A, 0 , A], is it and why?

cheers.

change {scaling_factors = [torch.ones(2, requires_grad=True).to(device) for _ in range(len(weights_to_be_quantized))]}

to

{scaling_factors = [torch.ones(2, requires_grad=True, device=device) for _ in range(len(weights_to_be_quantized))]}

works

hi, @vinsis

I wonder if you have tried on other datasets or networks? For example,
the listed case in https://github.com/czhu95/ternarynet (which seems to be the repo by the paper author).

I spent many time and couldn't recover the reported accuracy in the paper.