val-iisc / deligan

This project is an implementation of the Generative Adversarial Network proposed in our CVPR 2017 paper - DeLiGAN : Generative Adversarial Networks for Diverse and Limited Data. DeLiGAN is a simple but effective modification of the GAN framework and aims to improve performance on datasets which are diverse yet small in size.

DeLiGAN

This project is an implementation of the Generative Adversarial Network proposed in our CVPR 2017 paper - DeLiGAN : Generative Adversarial Networks for Diverse and Limited Data. Via this project, we make two contributions:

1. We propose a simple but effective modification of the GAN framework for settings where training data is diverse yet small in size.
2. We propose a modification of inception-score proposed by Salimans et al. Our modified inception-score provides a single, unified measure of inter-class and intra-class variety in samples generated by a GAN.

Dependencies

The code for DeLiGAN is provided in Tensorflow 0.10 for the MNIST and Toy dataset, and in Theano 0.8.2 + Lasagne 0.2 for the CIFAR-10 and Sketches dataset. This code was tested on a Ubuntu 14.04 workstation hosting a NVIDIA Titan X GPU.

Datasets

This repository includes implementations for 4 different datasets.

1. Toy (self generated unimodal and bimodal gaussians)
2. MNIST (http://www.cs.toronto.edu/~gdahl/mnist.npz.gz)
3. CIFAR-10 (https://www.cs.toronto.edu/~kriz/cifar.html)
4. Sketches (http://cybertron.cg.tu-berlin.de/eitz/projects/classifysketch/)

The models for evaluating DeLiGAN on these datasets can be found in our repo. The details for how to download and lay out the datasets can be found in src/datasets/README.md

Usage

Training DeLiGAN models

To run any of the models

• First download the datasets and store them in the respective sub-folder of the datasets folder (src/datasets/)
• To run the model on any of the datasets, go to the respective src folders and run the dg_'dataset'.py file in the respective dataset folders with two arguments namely, --data_dir and --results_dir. For example, starting from the top-level folder,

cd src/sketches 
python dg_sketches.py --data_dir ../datasets/sketches/ --results_dir ../results/sketches

• Note that the results_dir needs to have 'train' as a sub-folder.

Modified inception score

For example, to obtain the modified inception scores on CIFAR

• Download the inception-v3 model (http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz.) and store it in src/modified_inception_scores/cifar10/
• Generate samples using the model trained in the dg_cifar.py and copy it to src/modified_inception_scores/cifar10/
• Run transfer_cifar10_softmax_b1.py to transfer learn the last layer.
• Perform the modifications detailed in the comments in transfer_cifar10_softmax_b1.py and re-run it to evaluate the inception scores.
• The provided code can be modified slightly to work for sketches as well by following the comments provided in transfer_cifar10_softmax_b1.py

Parts of the code in this implementation have been borrowed from the Improved-GAN implementation by OpenAI (T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen. Improved techniques for training gans. In Advances in Neural Information Processing Systems, pages 2226–2234, 2016.)

Cite

@inproceedings{DeLiGAN17,
  author = {Gurumurthy, Swaminathan and Sarvadevabhatla, Ravi Kiran and R. Venkatesh Babu},
  title = {DeLiGAN : Generative Adversarial Networks for Diverse and Limited Data},
  booktitle = {Proceedings of the 2017 Conference on Computer Vision and Pattern Recognition},
  location = {Honolulu, Hawaii, USA}
 }

Q&A

Please send message to [email protected] if you have any query regarding the code.

deligan's Issues

how to use cpu to train for deli_gan

Hello,I am sorry to disturb you!I want to train deli_gan by cpu ,but there a lot of issues about it. should I modify the code ?

Mode Collapse for toy dataset?

Hi,

I was running your script in toy directory to reproduce results shown in paper, however ending up with mode collapse for both unimodal and multimodal data.

To deduce the reflection on initialization, I've run this test for 5 times on both datasets, results are pretty similar.

Any comments?

TypeError: ('An update must have the same type as the original shared variable (shared_var=W, shared_var.type=GpuArrayType<None>(float32, (False, True, False, False)), update_val=Elemwise{sub,no_inplace}.0, update_val.type=TensorType(float32, 4D)).', 'If the difference is related to the broadcast pattern, you can call the tensor.unbroadcast(var, axis_to_unbroadcast[, ...]) function to remove broadcastable dimensions.')

I got this error when I run the code in the folder sketches,but I can't find the solution to this error.

fixed batch size

if I want to apply deli-GAN's spirits to some other application like conditional GAN , so the batch size have to fixed? I think this is not convenient.

generating same sample

Hi, when I utilize the trained model to generate the samples, which the corresponding code is

#Uncomment this block for generating GAN samples from given model ''' f = np.load(args.results_dir + '/disc_params1180.npz') param_values = [f['arr_%d' % i] for i in range(len(f.files))] for i,p in enumerate(disc_params): p.set_value(param_values[i]) print("disc_params fed") f = np.load(args.results_dir + '/gen_params1180.npz') param_values = [f['arr_%d' % i] for i in range(len(f.files))] for i,p in enumerate(gen_params): p.set_value(param_values[i]) print("gen_params fed") samples=[] for i in range(50): sample_x = samplefun() samples.append(sample_x) samples = np.concatenate(samples,0) print(samples) #sys.exit() np.save(args.results_dir + '/samples50k.npy', samples) print("samples saved") sys.exit() '''

but for each iteration i, the image in the sample_x are diverse, but every sample_x is the same. what' wrong with it?

Results in dg_mnist.py

I've tried the mnist model. However, I only got some noises.....any comments?

Latent Space

Hi,
I noticed the paper said the trained latent space is a mixed Gaussian distribution with trainable variance and expectation:

In particular, we propose a reparameterization of the latent space as a Mixture- of-Gaussians model.

However, it seems that in the script the latent space applied here is a uniform distribution with trainable variance and expectation:
display_z = np.random.uniform(-1.0, 1.0, [batchsize, z_dim]).astype(np.float32)
I don't quite understand this inconsistency.

Some questions about the project

Two questions:
1.Do you find performance reduction if you increase the size of the generated image? Such as the dataset of the sketch? Can you give some suggestions if I want to increase the size of the results?
2.How to modify the network if I want to generate specific kind of images, just like the conditional GAN? Any suggestions?
Thanks!