xavierxiao / likelihood-regret

Official implementation for Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder at NeurIPS 2020

Likelihood-Regret

Official implementation of Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder at NeurIPS 2020.

Training

To train the VAEs, use appropriate arguments and run this command:

python train_pixel.py

Evaluation

To evaluate likelihood regret's OOD detection performance, run

python compute_LR.py

To evaluate likelihood ratio, run

python test_likelihood_ratio.py

To evaluate input complexity, run

python test_inputcomplexity.py

Above commands will save the numpy arrays containing the OOD scores for in-distribution and OOD samples in specific location, and to compute aucroc score, run

python aucroc.py

Pre-trained Models

You can download pretrained VAE models on FMNIST and CIFAR-10 here.

likelihood-regret's Issues

Batch_size

Hi, very interesting work!
I use your code and find that a bigger batch size cannot benefit the OOD results. Is this true?

Not Implementing IWAE?

Hi,

Thanks for your good work and repo. May I ask did you implement the exact IWAE as a tighter lower bound, in case I missed it?

See the reference:

Likelihood-Regret/train_pixel.py

Line 130 in 5517c9b

[z,mu,logvar] = netE(x)