kamnitsask / ssl_compact_clustering Goto Github PK

After communications with a user (thanks Shadab), we found that if this code is run using Tensorflow-2, training the model does not lead to learning/convergence. In fact, error goes super-high, eg 90% on mnist.

On his same settings, simply downgrading to Tensorflow 1.13 (where the code was written) solves this and everything works like normal.

Please use Tensorflow-1 for now (I ve tested till 1.13).

I will try to make time and update my projects to TF2 in the next few months (hopefully Dec'19 + Jan'20). But maintaining all projects takes quite some time so please excuse delays.

Thank you,
Konstantinos

Hi,

I am working on reproducing your work using pyTorch.

I have trouble with matrix inverse.

I saw that you use tf.matrix_inverse to compute matrix inverse.

However, I think that some I_uu - H_uu doesn't full rank so that that matrix is not invertible.

In tensorflow doc,

If a matrix is not invertible there is no guarantee what the op does. It may detect the condition and raise an exception or it may simply return a garbage result.

Is there any guarantee that I_uu - H_uu is always invertible??

Thanks,

From what I see in this line, the existence of tf.reduce_sum causes the tf.reduce_mean to divide by N instead of N^2 as stated in equation (9) in the paper.