is_training=False about im2latex-tensorflow

[ritheshkumar95]

Hey,

This issue seems peculiar to me.

Are you saying is_training=True returns the correct answer but is_training=False returns the wrong one?

[wh0]

Are you saying is_training=True returns the correct answer but is_training=False returns the wrong one?

Yes

[wh0]

I've found that the batch_norm wasn't working right in the training that went into the released weight files. The moving_mean and moving_variance tensors have initial values. We're going to try to update the code to carry out the batch_norm update operations during training. cc @mitar

[ritheshkumar95]

Right,

Sorry. I just noticed in tflib.contrib that batch_norm by default doesn't update moving means / averages. I thought it does by default.

Check this https://www.tensorflow.org/api_docs/python/tf/contrib/layers/batch_norm

[mingchen62]

I encountered the same issue with is_training=False. Not sure if @wh0 get a chance to update the code.

[wh0]

@mingchen62 I ended up not altering the training process at all.

[mingchen62]

All right. According to document, it looks like we will need to update "attention.py"
adding something like this:

update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)
with tf.control_dependencies(update_ops):
train_step =....

I will give it a try