b-remy / score-estimation-comparison Goto Github PK

Here is my plan for experimenting on images, and of course starting with MNIST.

• : Train a PixelCNN: this will act as our reference logp model, we'll see how much we can learn from it
• : Sampling from gradients: We want to test the different sampling methods for generating images
• : Train an AR-DAE: this will test the model, hopefully we will find some problems in the sampling
• : Train a SN-AR-DAE: Hopefully this should work well

My first experimentations are here:
https://github.com/b-remy/score-estimation-comparison/blob/master/notebooks/Flax_NCSNv3.ipynb

Once we have the framework working on simple 2d test distributions, we will need to check what happens on more complex datasets. For that we can grab some datasets from TF datasets.
We could start with MNIST, but for a paper, we might want to have a celeba example ;-) This issue is to document the choice of dataset and some of the tests run on them.

This first issue is to document the development/testing of using NF to learn the gradients of a prior distribution. To begin with, we can use the 2 moons distribution as our baseline

We need a small module/function that takes as an input a score function, and runs a chain to draw samples from the target distribution. At first, we can test that function on the analytic 2 moons distribution, with analytic gradients.

Implement a small function/module that implements a sampler based on Langevin dynamics. At first we can test that on the analytic 2 moons distribution.