ishmaelbelghazi / ali Goto Github PK
View Code? Open in Web Editor NEWAdversarially Learned Inference
License: MIT License
Adversarially Learned Inference
License: MIT License
In the decoder module. I found that z is sampled from N(0, 1), so where did you use the reparametrization trick described in formual (2) and (3) in the paper
I've been trying to reproduce your figures for semi-supervised learning on CIFAR-10 (19.98% with 1000 labels). This result is based on the technique proposed in Salimans et al. (2016), not SVMs. Is there any way you can include your code, or at least any changes to the hyperparameters in ali_cifar10.py
?
Thanks in advance for your help.
In the paper, it's mentioned that the input of discriminator is either joint pairs of q(x~,z) or p(x,z^). However, in the code, it seems (z,z^) and (x,x~) are concatenated and being sent as input to z_discriminator
and x_discriminator
respectively.
There seems to be a discrepancy here. Please let me know if I'm missing something.
Thanks.
Hi,
I was trying to reproduce the representation learning results of paper. Everything works fine except "preprocess_representations" script. It is leading to this error:
File "scripts/preprocess_representations", line 32, in preprocess_svhn
bricks=[ali.encoder.layers[-9], ali.encoder.layers[-6],
AttributeError: 'GaussianConditional' object has no attribute 'layers'
Any help would be appreciated.
Hello,I read the paper and the source code.And it mentioned 'The last three hidden layers of the encoder as well as its output are concatenated to form a 8960-dimensional feature vector.' in section 4.3 of the paper.Could you please tell me how to compute the dimension?Thanks very much
In the paper and code you used a set of 1x1 2D convolution layers to process the latent vector z in the discriminator. What was the motivation behind using 2D Convolutions versus fully connected layers or some other kind of convolutional layer? What other architectures did you try, and did you find success with any of those?
Hi @IshmaelBelghazi - I thought it might be useful to document some of the more useful updates in my fork that could be merged back in:
All of this is tested and working well for me. Feel free to steal anything you want, or let me know if you'd like me to prepare a merge request with anything you see that would be useful.
$ pip install -e ali
$ Directory 'ali' is not installable. File 'setup.py' not found.
Any ideas of the reason? Thanks!
Hi, I am so interested with your work, could you upload your pre-trained models (i.e., ImageNet) online ? Thanks!
Hi, any hint about how does batch norm update and why you do the following?
bn_updates = [(p, m * 0.05 + p * 0.95) for p, m in pop_updates]
Just trying to mimic the same behavior in Pytorch but not sure what are p
and m
and when this update happens during the execution.
I need to load a pretrained model with CPU only, but it appears the cuda arrays are serialized as such. Is there any way around this?
@vdumoulin I'm a bit confused about why are you using softplus activation** for the discriminator since in the paper formulas there no mention or at least I cannot recognize it.
**https://github.com/IshmaelBelghazi/ALI/blob/master/ali/bricks.py#L69-L72
BTW, @vdumoulin we met at your talk in the CVC during the break time :D
Could you possibly upload this or provide an external link?
Hi, I am replicating the code with PyTorch.
But I am not sure about the dropout behavior here. Seems that you apply dropout after all layers of the Discriminator (i.e. conv -> dropout -> bn -> dropout -> leaky relu -> dropout etc.), is that correct?
Also, do you apply any preprocessing on the input data? Seems that you rescale it to [0, 1]?
Thanks!
Hi, in the paper pseudocode, for Loss_d and Loss_g, the gradient ascent is turned into gradient descent by making the whole loss negative (negative signs in front of every term).
However, in the code, I don't see any of those negative signs. What am I missing?
Thank you!
Training goes well for me using the scripts in experiments
with the latest version of blocks, but then when I run any subsequent command that uses the generated model like scripts/sample
or scripts/reconstruct
, the command hangs indefinitely. My guess is that the deserialization is getting jammed up.
I can look into it more - not yet familiar with the new tar format - but curious if this might be a known issue.
I installed the current development version of fuel, but had some issue in fuel downloading.
$ fuel-download celeba 64
$ fuel-convert celeba 64
$ fuel-download celeba 64 --clear
The error message I got is:
fuel-download: error: unrecognized arguments: 64
if I remove 64, I got:
TypeError: init() got an unexpected keyword argument 'max_value'
Could someone please specify what version or commits of fuel and progressbar should I use? Thanks
I followed the same steps in the readme file, but when I run this line
$ THEANORC=theanorc python experiments/ali_cifar10.py
I get:
Traceback (most recent call last):
File "experiments/ali_cifar10.py", line 3, in <module>
from ali.utils import get_log_odds, conv_brick, conv_transpose_brick, bn_brick
ImportError: No module named ali.utils
In table 5 from the paper you state that the input size for D(x,z)
is 1024x1x1
which I think it's wrong after looking at the previous output sizes D(x)
and D(z)
. I think that should be 1536x1x1
.
Is that assumption correct?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.