jkdomoguen / uncertainty-aware-pytorch-implem

The pytorch implementation of paper "Uncertainty-Aware Attention for Reliable Interpretation and Prediction" by Jay Heo, Hae Beom Lee, Saehoon Kim, Juho Lee.

Uncertainty-Aware-Pytorch-Implem

The pytorch implementation of paper "Uncertainty-Aware Attention for Reliable Interpretation and Prediction" by Jay Heo, Hae Beom Lee, Saehoon Kim, Juho Lee. Based on its Tensorflow implementation from the original authors here

Requirements Python 3.7.x, Pytorch 1.1.x

Provided in dir dataset: Physionet in numpy format predicting mortatility risk Other dataset can be found here for Physionet for MIMIC-III dataset.

This implementation was able to achieve ~76% AUCROC on the provided evaluation dataset which is short of the ~78% claimed in the paper. The configuration follows exactly the configuration in the original implementation by the authors.

uncertainty-aware-pytorch-implem's Issues

about the implementation of sampling z

Hi, in the paper, they say that

we first sample random weights with dropout masks \omega and sample z such that z=g(x, \epsilon, \omega), \epsilon~N(0, I), with a pathwise derivative function g for reparameterization trick
(lines below Eq. 6)

and

we first sample dropout masks \omega (s) ∼ q_M(\omega|X, Y) and then sample z~p_θ(z|x^*, \omega)
(line below Eq. 7)

I have two questions:

1. Why do they ignore KL divergence of q(z) and p(z) in loss function (i.e. Eq. 6)? Why are they equivalent?
2. How do you implement 'first sample \omega, then sample z'? (I see that you sample z according to mu and sigma, but I don't what do they mean..)
   Thanks!