kozistr / awesome-gans

Awesome Generative Adversarial Networks with tensorflow

Awesome-GANs with Tensorflow

Tensorflow implementation of GANs (Generative Adversarial Networks)

WIP : This repo is about to be refactored & supporting tf 2.x.

Maybe some codes wouldn't work on master branch, because i'm just working on the branch.

Environments

Because of the image and model size, (especially BEGAN, SRGAN, StarGAN, ... using high resolution images as input), if you want to train them comfortably, you need a GPU which has more than 8GB.

But, of course, the most of the implementations use MNIST or CIFAR-10, 100 DataSets. Meaning that we can handle it with EVEN lower spec GPU than 'The Preferred' :).

Usage

Now on refactoring... All GAN training script can be run module-wisely like below. (WIP)

Install dependencies

You can also use conda, virtualenv environments.

$ python3 -m pip install -r requirements.txt

Train GANs

Before running the model, make sure that

1. downloading the dataset like CelebA, MNIST, etc what you want
2. In awesome_gans/config.py, there are several configurations, customize with your flavor!
3. running the model like below

$ python3 -m awesome_gans.acgan

DataSets

Supporting datasets are ... (code is in /awesome_gans/datasets.py)

• MNIST / Fashion MNIST
• CIFAR10 / 100
• CelebA/CelebA-HQ
• Pix2Pix
• DIV2K
• (more DataSets will be added soon!)

Repo Tree

│
├── awesome_gans (source codes & eplainations & results & models) 
│        │
│        ├── acgan
│        │    ├──gen_img (generated images)
│        │    │     ├── train_xxx.png
│        │    │     └── train_xxx.png
│        │    ├── model  (pre-trained model file)
│        │    │     └── model.txt (google-drive link)
│        │    ├── __init__.py
│        │    ├── __main__.py
│        │    ├── model.py (gan model)
│        │    ├── train.py (gan trainer)
│        │    ├── gan_tb.png   (tensorboard loss plot)
│        │    └── readme.md    (results & explainations)
│        ├── config.py         (configurations)
│        ├── modules.py        (networks & operations)
│        ├── utils.py          (auxiliary utils)
│        ├── image_utils.py    (image processing)
│        └── datasets.py       (dataset loader)
├── CONTRIBUTING.md
├── Makefile   (for linting the codes)
├── LICENSE
├── README.md  (Usage & GAN paper list-up)
└── requirements.txt

Papers & Codes

Here's the list-up for tons of GAN papers. all papers are sorted by alphabetic order.

Start

Here's the beginning of the GAN.

Name	Summary	Paper	Code
GAN	Generative Adversarial Networks	[arXiv]	[code]

Theory & Concept

Here for the theories & concepts of the GAN.

Name	Summary	Paper	Code	Official Code
ACGAN	Auxiliary Classifier Generative Adversarial Networks	[arXiv]	[code]
AdaGAN	Boosting Generative Models	[arXiv]	[code]
bCR	Improved Consistency Regularization for GANs	[arXiv]	[code]
BEGAN	Boundary Equilibrium Generative Adversarial Networks	[arXiv]	[code]
BGAN	Boundary-Seeking Generative Adversarial Networks	[arXiv]	[code]
BigGAN	Large Scale GAN Training for High Fidelity Natural Image Synthesis	[arXiv]	[code]
CGAN	Conditional Generative Adversarial Networks	[arXiv]	[code]
CoGAN	Coupled Generative Adversarial Networks	[arXiv]	[code]
ConSinGAN	Improved Techniques for Training Single-Image GANs	[WACV21]	[code]	[official]
DCGAN	Deep Convolutional Generative Adversarial Networks	[arXiv]	[code]
DRAGAN	On Convergence and Stability of Generative Adversarial Networks	[arXiv]	[code]
EBGAN	Energy-based Generative Adversarial Networks	[arXiv]	[code]
EqGAN-SA	Improving GAN Equilibrium by Raising Spatial Awareness	[arXiv]	[code]	[official]
f-GAN	Training Generative Neural Samplers using Variational Divergence Minimization	[arXiv]	[code]
GP-GAN	Towards Realistic High-Resolution Image Blending	[arXiv]	[code]
Softmax GAN	Generative Adversarial Networks with Softmax	[arXiv]	[code]
GAP	Generative Adversarial Parallelization	[arXiv]	[code]
GEGAN	Generalization and Equilibrium in Generative Adversarial Networks	[arXiv]	[code]
G-GAN	Geometric GAN	[arXiv]	[code]
InfoGAN	Interpretable Representation Learning by Information Maximizing Generative Adversarial Networks	[arXiv]	[code]
LAPGAN	Laplacian Pyramid Generative Adversarial Networks	[arXiv]	[code]
LSGAN	Loss-Sensitive Generative Adversarial Networks	[arXiv]	[code]
MAGAN	Margin Adaptation for Generative Adversarial Networks	[arXiv]	[code]
MRGAN	Mode Regularized Generative Adversarial Networks	[arXiv]	[code]
MSGGAN	Multi-Scale Gradients for Generative Adversarial Networks	[arXiv]	[code]
PGGAN	Progressive Growing of GANs for Improved Quality, Stability, and Variation	[arXiv]	[code]	[official]
RaGAN	The relativistic discriminator: a key element missing from standard GAN	[arXiv]	[code]
SeAtGAN	Self-Attention Generative Adversarial Networks	[arXiv]	[code]
SphereGAN	Sphere Generative Adversarial Network Based on Geometric Moment Matching	[CVPR2019]	[code]
SGAN	Stacked Generative Adversarial Networks	[arXiv]	[code]
SGAN++	Realistic Image Synthesis with Stacked Generative Adversarial Networks	[arXiv]	[code]
SinGAN	Learning a Generative Model from a Single Natural Image	[arXiv]	[code]	[official]
StableGAN	Stabilizing Adversarial Nets With Prediction Methods	[arXiv]	[code]
StyleCLIP	Text-Driven Manipulation of StyleGAN Imagery	[arXiv]	[code]	[official]
StyleGAN	A Style-Based Generator Architecture for Generative Adversarial Networks	[arXiv]	[code]	[official]
StyleGAN2	Analyzing and Improving the Image Quality of StyleGAN	[arXiv]	[code]	[official]
StyleGAN2 ADA	StyleGAN2 with adaptive discriminator augmentation	[arXiv]	[code]	[official]
StyleGAN3	Alias-Free Generative Adversarial Networks	[arXiv]	[code]	[official]
StyleGAN-XL	Scaling StyleGAN to Large Diverse Datasets	[arXiv]	[code]	[official]
TripleGAN	Triple Generative Adversarial Networks	[arXiv]	[code]
UGAN	Unrolled Generative Adversarial Networks	[arXiv]	[code]
U-Net GAN	A U-Net Based Discriminator for Generative Adversarial Networks	[CVPR20]	[code]	[official]
WGAN	Wasserstein Generative Adversarial Networks	[arXiv]	[code]
WGAN-GP	Improved Training of Wasserstein Generative Adversarial Networks	[arXiv]	[code]

Applied Vision

Here for the GAN applications on Vision domain, like image-to-image translation, image in-painting, single image super resolution , etc.

Name	Summary	Paper	Code	Official Code
3D GAN	3D Generative Adversarial Networks	[MIT]	[code]
AnycostGAN	Anycost GANs for Interactive Image Synthesis and Editing	[arXiv]	[code]	[official]
CycleGAN	Unpaired img2img translation using Cycle-consistent Adversarial Networks	[arXiv]	[code]
DAGAN	Instance-level Image Translation by Deep Attention Generative Adversarial Networks	[arXiv]	[code]
DeblurGAN	Blind Motion Deblurring Using Conditional Adversarial Networks	[arXiv]	[code]
DualGAN	Unsupervised Dual Learning for Image-to-Image Translation	[arXiv]	[code]
DRIT/++	Diverse Image-to-Image Translation via Disentangled Representations	[arXiv]	[code]	[official]
EdgeConnect	Generative Image Inpainting with Adversarial Edge Learning	[arXiv]	[code]	[official]
ESRGAN	Enhanced Super-Resolution Generative Adversarial Networks	[arXiv]	[code]
FastGAN	Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesis	[arXiv]	[code]	[official]
FUNIT	Few-Shot Unsupervised Image-to-Image Translation	[arXiv]	[code]	[official]
CA & GA	Generative Image Inpainting w/ Contextual Attention & Gated Convolution	[CVPR2018], [ICCV2019]	[code]	[official]
HiFaceGAN	Face Renovation via Collaborative Suppression and Replenishment	[arXiv]	[code]
MUNIT	Multimodal Unsupervised Image-to-Image Translation	[arXiv]	[code]	[official]
NICE-GAN	Reusing Discriminators for Encoding	[arXiv]	[code]	[official]
PSGAN	Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer	[arXiv]	[code]	[official]
SpAtGAN	Generative Adversarial Network with Spatial Attention for Face Attribute Editing	[ECCV2018]	[code]
SalGAN	Visual Saliency Prediction Generative Adversarial Networks	[arXiv]	[code]
SRFlow	Super-Resolution using Normalizing Flow	[arXiv]	[code]	[official]
SRGAN	Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network	[arXiv]	[code]
SRResCGAN	Deep Generative Adversarial Residual Convolutional Networks for Real-World Super-Resolution	[arXiv]	[code]	[official]
StarGAN	Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation	[arXiv]	[code]	[official]
StarGAN V2	Diverse Image Synthesis for Multiple Domains	[arXiv]	[code]	[official]
StyleGAN-V	A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2	[arXiv]	[code]	[official]
TecoGAN	Learning Temporal Coherence via Self-Supervision for GAN-based Video Generation	[arXiv]	[code]	[official]
TextureGAN	Controlling Deep Image Synthesis with Texture Patches	[arXiv]	[code]
TUNIT	Rethinking the Truly Unsupervised Image-to-Image Translation	[arXiv]	[code]	[official]
TwinGAN	Cross-Domain Translation fo Human Portraits	[github]	[code]
UNIT	Unsupervised Image-to-Image Translation Networks	[arXiv]	[code]	[official]
XGAN	Unsupervised Image-to-Image Translation for Many-to-Many Mappings	[arXiv]	[code]
Zero-DCE	Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement	[CVPR20]	[code]	[official]

Applied Audio

Here for the GAN applications on Audio domain, like wave generation, wave to wave translation, etc.

Name	Summary	Paper	Code	Official Code
AAS	Adversarial Audio Synthesis	[arXiv]	[code]
BeatGAN	Generating Drum Loops via GANs	[arXiv]	[code]
GANSynth	Adversarial Neural Audio Synthesis	[arXiv]	[code]
MuseGAN	Multi-track Sequential GANs for Symbolic Music Generation and Accompaniment	[arXiv]	[code]
SEGAN	Speech Enhancement Generative Adversarial Network	[arXiv]	[code]
StarGAN-VC	Non-parallel many-to-many voice conversion with star generative adversarial networks	[arXiv]	[code]
TempoGAN	A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow	[arXiv]	[code]
Parallel WaveGAN	A fast waveform generation model based on GAN with multi-resolution spectrogram	[arXiv]	[code]
WaveGAN	Synthesizing Audio with Generative Adversarial Networks	[arXiv]	[code]

Applied Others

Here for the GAN applications on other domains, like nlp, tabular, etc.

Name	Summary	Paper	Code	Official Code
AnoGAN	Unsupervised Anomaly Detection with Generative Adversarial Networks	[arXiv]	[code]
CipherGAN	Unsupervised Cipher Cracking Using Discrete GANs	[github]	[code]
DiscoGAN	Discover Cross-Domain Generative Adversarial Networks	[arXiv]	[code]
eCommerceGAN	A Generative Adversarial Network for E-commerce	[arXiv]	[code]
PassGAN	A Deep Learning Approach for Password Guessing	[arXiv]	[code]
SeqGAN	Sequence Generative Adversarial Networks with Policy Gradient	[arXiv]	[code]
TAC-GAN	Text Conditioned Auxiliary Classifier Generative Adversarial Network	[arXiv]	[code]

Useful Resources

Here for the useful resources when you try to train and stable a gan model.

Name	Summary	Link
GAN Hacks	a bunch of tips & tricks to train GAN stable	github

Note

Any suggestions and PRs and issues are WELCOME :)

Author

HyeongChan Kim / @kozistr

awesome-gans's Issues

Incorrect conjugate function in f-GAN implementation

Describe the bug
In the file "fgan_model.py":
(1) Line 209, the conjugate function of Total-Variation in your code is supposed to be the same as the activation function. Since in your implementation, the conjugate function should be f*(g(x)) where x is the input. Thus, for Total Variation, f*(g(x))=g(x) rather than x;
I think it should be: -torch.mean(torch.tanh(x) / 2.) ?
(2) The conjugate of Jenson-Shannon is also incorrect? Line159;
I think it should be: -torch.mean(x + torch.log(1. + torch.exp(-x))) - torch.log(torch.tensor(2.))?
(3) The conjugate of Hellinger is also incorrect? Line175.
I think it should be: -torch.mean((1. - torch.exp(x)) / (torch.exp(x)))?

[feature] pip

run srgan on my own photo

hi @kozistr thank you for your hard and great work
could you please explain to me how to use your srgan to upscale my own photo
thank you

Performance issue in /awesome_gans/pggan (by P3)

Hello! I've found a performance issue in /awesome_gans/pggan/pggan_train.py: with tf.Session(config=config) as s(here) is repeatedly called in the loop for idx, n_pg in enumerate(pg)(here).

tf.Session being defined repeatedly could lead to incremental overhead. If you define tf.Session out of the loop and pass tf.Session as a parameter to the loop, your program would be much more efficient. Here is the Stack Overflow post to support it.

Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.

SRGAN

In run-time, loss became NaN...

now on re-modeling & building for real SR images...

out of memory error

@kozistr When I try to run your code, then I am getting a out of memory error. I looked at it and apparently, when you are loading the data, you are storing all images in memory at once, which is causing this error. Can you help me in this matter?

[rework] rework the baseline code, tensorflow 1.x to pytorch

Currently, all of the codes are based on tensorflow 1.x, which is legacy.

so, rework all codes to pytorch code, instead of tensorflow 2.x

LAPGAN

In the lapgan_train.py, the 56th line says:
batch_images = batch_images.astype(np.float32) / 225.
Why it is 225 instead of 255?

SGAN

Image generation has a problem. (maybe, entire model...)

infoGAN list_attr_celeba.txt

Could you show your "list_attr_celeba.txt" used in the infogan folder? I have made the txt file myself, but it doesn't work well, plz!

stargan

i can not understand the difference between batch_a and batch_b in stargan

CycleGAN

It's just not worked well... It'll be fixed soon

Figure out CGAN's condition label

Hi @kozistr, Thanks for your great work!

In your CGAN implementation, you feed G and D with the same real label from the dataset as the condition.However, I have a question about this since several other implementations feed G with a sampled label as the condition.

I have seen several CGAN implementations. There are two ways to use the condition labels:

1. feed G with a random label, feed D with the real label
   • Kera-GAN
   • PyTorch-GAN
   • pytorch-MNIST-CelebA-cGAN-cDCGAN
   • tensorflow-MNIST-cGAN-cDCGAN
2. feed both G and D with the same real label
   • GAN
   • tensorflow-generative-model-collections
   • generative-models
   • pytorch-generative-model-collections

Here are my questions:

• Which way is used by the CGAN paper in 2014?
• When training the CGAN, should we use the same condition label for both G and D?

The same issue has been asked here. I hope we can figure out it together.

Coverage with tfutil.py

now on going ~

[rework] rework the code quality

Currently, there're lots of duplicates of codes. and hard to use and read.

So, modulizing and refactoring the code is needed.

Migrate LGTM.com installation from OAuth to GitHub App

Hi There,

This project is still using an old implementation of LGTM's automated code review, which has now been disabled. To continue using automated code review, and receive checks on your Pull Requests, please install the GitHub App on this repository.

Thanks,
The LGTM Team

GAN image generation problem

Currently, there're GANs that don't work properly with some reasons that i didn't find yet, but not in the past,

• EBGAN

Need to be fixed. Any helps or suggestions are welcome :)

[rework] rework visual methods

Adding visual contents would be good #14

CoGAN

Hello,

I am trying to run CoGAN, but I get the following error below:

Traceback (most recent call last):
File "cogan_train.py", line 131, in
main()
File "cogan_train.py", line 42, in main
model = cogan.CoGAN(s)
File "/home/maria/Documents/Awesome-GANs/CoGAN/cogan_model.py", line 166, in init
self.build_cogan() # build CoGAN model
File "/home/maria/Documents/Awesome-GANs/CoGAN/cogan_model.py", line 224, in build_cogan
self.g_1 = self.generator(self.z, self.y, share_params=False, reuse=False, name='g1')
File "/home/maria/Documents/Awesome-GANs/CoGAN/cogan_model.py", line 196, in generator
x = tf.layers.dense(x, 7 * 7, reuse=True, name='gen-dense-reshape')
File "/home/maria/anaconda2/envs/anaconda3gpu/lib/python3.6/site-packages/tensorflow/python/layers/core.py", line 250, in dense
return layer.apply(inputs)
File "/home/maria/anaconda2/envs/anaconda3gpu/lib/python3.6/site-packages/tensorflow/python/layers/base.py", line 671, in apply
return self.call(inputs, *args, **kwargs)
File "/home/maria/anaconda2/envs/anaconda3gpu/lib/python3.6/site-packages/tensorflow/python/layers/base.py", line 559, in call
self.build(input_shapes[0])
File "/home/maria/anaconda2/envs/anaconda3gpu/lib/python3.6/site-packages/tensorflow/python/layers/core.py", line 137, in build
trainable=True)
File "/home/maria/anaconda2/envs/anaconda3gpu/lib/python3.6/site-packages/tensorflow/python/layers/base.py", line 458, in add_variable
trainable=trainable and self.trainable)
File "/home/maria/anaconda2/envs/anaconda3gpu/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 1203, in get_variable
constraint=constraint)
File "/home/maria/anaconda2/envs/anaconda3gpu/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 1092, in get_variable
constraint=constraint)
File "/home/maria/anaconda2/envs/anaconda3gpu/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 425, in get_variable
constraint=constraint)
File "/home/maria/anaconda2/envs/anaconda3gpu/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 394, in _true_getter
use_resource=use_resource, constraint=constraint)
File "/home/maria/anaconda2/envs/anaconda3gpu/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 760, in _get_single_variable
"reuse=tf.AUTO_REUSE in VarScope?" % name)
ValueError: Variable generator-g1/gen-dense-reshape/kernel does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=tf.AUTO_REUSE in VarScope?

Any help would be greatly appreciated! Thank you in advance.

Update Awesome-GAN with new lists

Is it possible for you to have an annual review of new GANs that arrives on the scene through https://github.com/nightrome/really-awesome-gan?

P.S. it would be nice if you can draw a diagram for each of the GANs and explain how they differ, and maybe create a generalized GAN interface so that people can hot-swap whatever CNNs they would like to use from https://github.com/CeLuigi/models-comparison.pytorch/wiki/Accuracy-vs-Computational-complexity for more diverse designs.

DataSet Processing Stuffs

Currently, using h5py for saving images.
Later, it'll be changed to TFRecord, overall.

And kinda image normalize like dividing into 255.0, it'll be also removed from datasets.py
Or by option :).

Anyway, in short, datasets.py reformed soon!

[rework] rework w/ paper list

• Select some of the papers which have impacts and important as i think,
• Summary all of the papers within 3 lines!
• Grouping the papers w/ following characteristic

Following instructions in README.md to train a GAN does nothing

Describe the bug
Trying to follow the README.md and code does nothing.

To Reproduce
Steps to reproduce the behavior:

1. Follow README.md instructions to train GANs

python3 -m awesome_gans.acgan

3. Nothing happens

Expected behavior
GAN should train

Additional context
This isn't entirely unexpected since the code for awesome_gans.acgan (i.e. acgan/__main__.py) is essentially blank as shown below. I'm wondering if this ever worked, or if something is very out of date.

from awesome_gans.config import parse_args
from awesome_gans.utils import set_seed

def main():
    config = parse_args()

    set_seed(config.seed)

    if config.mode == 'train':
        pass
    elif config.mode == 'inference':
        pass
    else:
        raise ValueError()


main()

PGGAN generator has some bug?

I find PGGAN can not train.
When pg ==1, generator make the fake image which shape is [-1,16,16, 3].
And i find reason: because generator has upsample twice on line of block the pggan_model.py .
repair

x = block(x, nf(i + 1), name="1")
# x = block(x, nf(i + 1), name="2")
x = tf.layers.conv2d(x, filters=get_out_channels(i + 1), kernel_size=3, strides=1, padding='SAME',
                        kernel_initializer=w_init, kernel_regularizer=w_reg, bias_initializer=b_init,
                        name='gen_n_%s_conv2d-%d' % (name, x.get_shape()[1]))
x = tf.nn.leaky_relu(x)
x = pix_norm(x)

Re-Run all GAN codes...

• For better results
• For better codes
• For checking fully working codes
• etc...