Text to Image generation

March 10, 2018

* created inverse gans

March 19, 2018

* added is_cuda check for train.py
* using --split=2 for inference (testing set)
* additional loss used by the original code author:
	g_loss = criterion(outputs, real_labels) \
					 + self.l2_coef * l2_loss(activation_fake, activation_real.detach()) \
					 + self.l1_coef * l1_loss(fake_images, right_images)

GAN losses

* forward GAN: 
	* disc_loss = real_loss + fake_loss + wrong_loss(cls)(wrong image + right embedding)
	* gen_loss = g_loss
* inverse GAN: 
	* disc_loss = real_loss + fake_loss + wrong_loss(cls)(wrong embedding + right image)
	* gen_loss = criterion(outputs, real_labels) # that's it?

Questions:

* cls: what would cls loss be for disc_loss in inverse GAN?

TODO:

prepare dataset COCO (dataloader function)
generate word embeddings (using skip thought or gensim?)
GAN for image generation from word embeddings
GAN for caption generation
Cycle GAN structure

csc2548's Projects

char-cnn-rnn

Learning Deep Representations of Fine-grained Visual Descriptions

cyclegan

Software that can generate photos from paintings, turn horses into zebras, perform style transfer, and more.

ganhacks

starter from "How to Train a GAN?" at NIPS2016

icml2016

Generative Adversarial Text-to-Image Synthesis

proposal

csc2548 project proposal

pytorch-cyclegan-and-pix2pix

Image-to-image translation in PyTorch (e.g. horse2zebra, edges2cats, and more)

tac-gan

A Tensorflow implementation of the Text Conditioned Auxiliary Classifier Generative Adversarial Network for Generating Images from text descriptions (https://arxiv.org/abs/1703.06412)