chrischute / real-nvp Goto Github PK

Hello,
Thanks for your elegant implementation. But I find it hard to understand what 'alt_order=True' means in squeeze_2x2 function. The tensor operations are really comfusing. Could you explain it in a more intutive way? And does it corresponds to any text in the origin paper?

Thanks.

I'm not sure if I'm misunderstanding the code, but in line 49 in coupling_layer.py, why is the masking applied before we take exp_s? Wouldn't s.exp() just set the values that are 0 to 1 in this case, meaning that no part of the affine scaling is masked out?

Hello,
could you plz explain, why double the input after the batchNorm and concat [input, -input]?

thanks

Hello! Thanks for implementing the paper. After comparing the text and your code I haven't found factorization feature. Is it there?

In coupling_layer, the reverse procedure is
if reverse:
x = x * inv_exp_s - t,
else:
x = (x + t) * exp_s

Why do you use -t for these two equations?

Best wishes

Hi,

Thank you so much for the wonderful code. I am new to this area and I want to use RNVP method to learn the joint probability of digit images. I tried MNIST dataset and the samples I got are rotated digits. For example, I would get 3 and E. Some images is vertically symmetrical.

It seems like there is a data augmentation process. However, I do not know why it happens. Hope this question will not bother you too much.

Hi, I wonder why k is used to compute the gauss probability in the loss function

Thanks for a great job and your implementation of ReavNVP.
I think there is a mistake in the implementation of reverse pass of RealNVP, since inversion of RealNVP layer should give $x_2 = (y_2 - t(y_1)) \odot \exp(-s(y_1))$, i.e in the line:

Hi, I tried to follow the instruction and run the training example but at the end of Epoch0 I got the following error message. I followed the Conda environment set-up in the instruction. I am wondering if you know what caused this issue. Thanks!

Traceback (most recent call last):
File "train.py", line 152, in
main(parser.parse_args())
File "train.py", line 63, in main
test(epoch, net, testloader, device, loss_fn, args.num_samples)
File "train.py", line 132, in test
torchvision.utils.save_image(images_concat, 'samples/epoch_{}.png'.format(epoch))
File "/home/mx83/anaconda3/envs/rnvp/lib/python3.7/site-packages/torchvision/utils.py", line 102, in save_image
ndarr = grid.mul(255).clamp(0, 255).byte().permute(1, 2, 0).cpu().numpy()
RuntimeError: PyTorch was compiled without NumPy support

Thanks for your work!
But I have a question about the value of loss, the loss of my trained model after 100 epochs is 7.33e+3 while the loss of the first epoch is 9.91e+3, is it unnormal? And is it means the likelihood is -loss, e.g., -9.91e+3?
By the way, as the loss decrease, bpd is also decrease from bpd=4.65 to bpd=3.44.
Can you tell me the value of your loss?

Hi, can I ask what is the cuda version for this repo?

Hello, I have a question for a function _pre_process.

In the original paper, author preprocessed CIFAR this way. y = alpha + (1 - alpha) * x / 256 (alpha = 0.05, *: elementwise multiplication)

In your code, formula seems a bit different from the original paper like this. y = log({0.5 + x/256 * alpha - alpha / 2} / {0.5 - x/256 * alpha + alpha / 2}). (alpha = 0.9)

I want to know if there is a reason you used different preprocessing.

Thank you for wonderful code and sorry for my pool English.

(I think it's because log(x / 1-x) is inverse function of sigmoid, so I close this issue)

Thank you for your code.

But, I want to ask why we need 'double_after_norm' in CHESSBOARD mask mode.

Thanks for an amazing open source version of RealNVP.

The forward pass for coupling layer computes

While RealNVP suggested

I suspect the difference in performance in negligible, but I felt it might be worth sharing.

I am training your code on cifar10 dataset, the loss is very high.. it feels kinda odd.
is there anything wrong with the code? Could you please check once?

the avg loss per epoch is in the range of 7k-8k, though I am at the initial epochs, still, it seems a bit odd..
I am fairly new in this area.. so I would appreciate your help.