mhaut / angulargrad Goto Github PK

Hi, I'm a machine learning engineer working in medical data company in south korea.

I found your paper and thought that it was state-of-the-art. I was totally impressed by your brilliant idea, so I wanted to make them available in Tensorflow. Now, I'm implementing it as tensorflow optimizer. In order to do so, I had to fully understand and analyze your codes. While reading codes, a question came to mind during the analysis and I decided to post an issue.

Whether its type is boolean or tensor is not important. ans1[0] == "False" cannot be True and always be False. This consequence makes codes below not approchable.

except:
    if (ans1[0] == "False"):
        min = angle
        diff = abs(previous_grad - grad)
        final_cos_theta = cos_theta.clone()

I'm not familiar with Pytorch. There could be possibility that I misunderstood the codes so i would appreciated it if you could explain this codes.

I am trying to reproduce the results of this work, and I encountered a methodological issue regarding angles computation.

The tangent of the angle between consecutive gradients is here computed as abs((previous_grad - grad) / (1 + previous_grad * grad)):

However, the result of this operation is not a scalar, but a vector with the same shape as grad. What is the interpretation of such quantity? The paper states that this is the tangent of the angle between consecutive gradients, but it does not seem to be the case.

Thank you in advance!