Can not reproduce results of Uncertainty_Demo_MNIST.ipynb about astronn

[henrysky]

Hi, the main issue is astroNN built-in data normalizer ignored mode=255 due to this faulty commit f8fb024 lead to the normalizer does nothing to normalize MNIST images and blow up the gradient. I am kinda still on holiday and will go back to research work on the coming Monday so the bug will be fully patched next week probably.

But I have updated some codes in the latest commit and there are some workarounds need to be done in your Jupyter Notebook as I do not want to modify the notebook yet.

1. add line net.mc_num = 25 after net = MNIST_BCNN() due to a performance issue, so do less Monte Carlo passes as a workaround
2. change pred, pred_std = net.test(x_test[test_idx]) topred, pred_std = net.test_old(x_test[test_idx]) due to the test() refers to the new fast MC inference on GPU now which turns out not handling classification task correctly and the old test() is renamed to test_old()
3. change pred_rot, pred_rot_std = net.test(test_rot) topred_rot, pred_rot_std = net.test_old(test_rot) for the same reason

This issue will remain open until the issue is fully resolved

To-do list for me:

• Add test cases to prevent similar issues (check Nan especially) Done!!
• The losses now have some kind of performance issue (Painfully slow even on GPU, definitely some operation(s) are being ran on CPU for some reasons) 50% Done!!
• The new accelerated test() for BNN is not handling classification task correctly (and add test case!!!) Done!!

[henrysky]

It should have fully resolved, no modification in the Uncertainty_Demo_MNIST.ipynb is needed

[snavavf]

Thanks for the quick update!
Now I can get reasonable loss from the second cell. Great.

However, in the third cell (Test the neural network on random MNIST images),
the total uncertainty (entropy) I got are all 1.0.

As in the following link
https://i.imgur.com/VaVfdsb.jpg

Could you suggest why?
Thanks!

[henrysky]

I acknowledge the issue.

My apology, I use regression only for my research so classification-related things are not tested regularly, the current continuous integration test cases only make sure things run without error but not reasonable result. I am looking into it.