musyoku / adversarial-autoencoder

Chainer implementation of adversarial autoencoder (AAE)

Adversarial AutoEncoder

• Code for the paper

Requirements

• Chainer 2+
• Python 2 or 3

Incorporating Label Information in the Adversarial Regularization

run semi-supervised/regularize_z/train.py

We trained with a prior (a mixture of 10 2-D Gaussians or Swissroll distribution) on 10K labeled MNIST examples and 40K unlabeled MNIST examples.

Supervised Adversarial Autoencoders

run supervised/learn_style/train.py

Semi-Supervised Adversarial Autoencoders

run semi-supervised/classification/train.py

data	#
labeled	100
unlabeled	49900
validation	10000

Validation accuracy at each epoch

Analogies

Unsupervised clustering

run unsupervised/clustering/train.py

16 clusters

32 clusters

Dimensionality reduction

run unsupervised/dim_reduction/train.py

run semi-supervised/dim_reduction/train.py

adversarial-autoencoder's Issues

Reproducing quantitative results

Hi musyoku,
Thank you very much for the code. I ran your implementation and was able to get the reported results in README.md.
However my question is, have you been able to reach an error rate of 1.90 (±0:10)% for MNIST 100 labels (as reported in original paper)? Even after 5000 epochs the lowest I get is somewhat around 5%.

Can't get the correct error rates

Hey

For the semi-supervised version (100 labels), the current version gets around 85% accuracy.
Which is different from the graph in the readme.

Here is the resultant semi-supervised/dim_reduction/results.csv
result.txt

The performance of the unsupervised dimensionality reduction also seems to have decreased.
Here's the visualization using the current version. The separation is worse than the visualization shown in readme. The error rate on the training data for this visualization is ~40%. The error rate reported in the original paper is ~13.5%.

Thanks!

dependency environment problem

Hello! I use anaconda spyder to compile your perfect project, but I meet some problems. I think it's about my compile environment. I use win7 64bit, and Chainer version is 2.01, cupy version 4.1, cuda 9.0, cudnn 7. There are always some problems about the carray.cuh of cupy. Could you please tell me your compile environment and do u know what about this problem?

Semi-supervised dimensionality reduction visualization with z dimension > 2

Quoting the paper "Once the network is trained, in order to visualize the 10D learnt
representation, we use a linear transformation to map the 10D representation to a 2D space such that the cluster heads are mapped to the points that are uniformly placed on a 2D circle.". With W_c = 10I we are simply adding the softmax y output to the the style parameters. I am not sure about the implementation of this linear transformation. Can you please provide some hint to implement this.

Need training data to run train.py

Would you please provide a link to download mnist data in the your required format?

init_scope missing in Module

Since your last commit I get

File "run/unsupervised/clustering/train.py", line 180, in
main()
File "run/unsupervised/clustering/train.py", line 29, in main
model = Model()
File "/Users/alessandro/adversarial-autoencoder/run/unsupervised/clustering/model.py", line 21, in init
nn.Tanh(),
File "/Users/alessandro/adversarial-autoencoder/aae/nn.py", line 250, in init
self.add(*layers)
File "/Users/alessandro/adversarial-autoencoder/aae/nn.py", line 253, in add
with self.init_scope():
AttributeError: 'Module' object has no attribute 'init_scope'

How to realize Unsupervised Clustering with Adversarial Autoencoders

Hi, I follow the instruction described in paper "Adversarial Autoencoders" to realize the Unsupervised Clustering with Adversarial Autoencoders(section 6). But i didn't get the right cluster result. I want to konw if you have already have the cluster net. Could you give me some useful infos and i could finsh it. Thanks

How can I set the GPU to 1?

Hello,

This is a great contribution! Thank you for the work.

I have two GPUs and would like to use GPU 1 to run the code. GPU0 is weak and only for my display. How can I do this with your code?

Thank you,

Amir

A question about cluster heads cost

Hello! Recently, I am studying one of your code about adversarial-autoencoder. I don't know the way that you define the starting labels and ending labels which appears in aae_dim_reduction.py. Could you tell me why you define them like that?

Errors while running codes

Hi,

I get the following error message when I try to run supervised/train.py

I want to run it successfully. Thank you in advance!

The code for SVHN dataset.

Recently, I am studying one of your codes for adversarial-autoencoder. I am supervised at its excellent performance on mnist. Since the original paper also makes the experiment for svhn dataset. I don't know whether you have the code for such dataset. Thank you very much for your replying!

Hi, question about unsupervised clustering

The paper mentioned that "The first image in each row shows the cluster heads, which are digits generated by fixing the style variable to zero and setting the label variable to one of the 16 one-hot vectors. The rest of the images in each row are random test image that have been categorized into the corresponding category based on q(y|x)"
I am so confused that are the rest of the images from the reconstruction images or from the validation dataset that just clustered into corresponding categories???

Why did you choose the softmax cross entropy ?

Hi ,Thanks for your code , I am confused about why you choose the softmax cross entropy , I also tried this loss ,unfortunately , it did work but not so good , when I regularized the z representation space into a gaussian distribution using the softmax cross entropy , I got a off-centre distribution ,like the following:

you can see gaussian expectation is about (-2, -2.5) rather than (0 , 0) ,which quitely confused me.
I can't explain it from theory, and you can see the equilibrium between the generator and discriminator,

D and G have reached the expected value about log2 and log4 respectively, and I don't know why the equilibrium can be found, In general standpoint, discriminator can be easily judge the sample generated from generator, and I have been well check the code , it have no bug or type the wrong variable. I don't know what's going on and what't wrong with it ?

Errors while running codes

Hi,

I get the following error message when I try to run unsupervised/clustering/train.py

And I get the follwing message when I try to run supervised/learn_style/train.py

I am able to run semi-supervised/regularize_z/train.py successfully though. Thanks in advance!

Is there any difference on semi-supervised decoders' inputs?

Dim_reduction adds a representation part before decoder compared to normal semi-supervised model. However, the inputs of decoders are all addition between tensors, is there any difference? I think normal semi-supervised model's decoder's input should be addition of dimensions (dim_z + dim_y), but I'm not sure.

undocumented dependency

Hi, i just tried your code and found out that it requires scikit-learn package,
please update the readme file accordingly.

Thank you very much for your contribution.