sxhxliang / biggan-pytorch Goto Github PK

hello:
Thanks for your share. It is amazing. When I run the code , I find the model choices are just 'sagan' and 'qgan‘. So the code is about the two model, not "biggan"? Or Other? Would like your reply. Thanks a lot.

Hello, thank you for the wonderful repo. I lack any hardware to train one of these GANs to the quality that you display in the README. Could you please provide a link to the latest checkpoint for that model?

Have you tried to generate higher resolution images? (512,1024,higher)

Thanks to your share code.And I always get the error:TypeError: forward() missing 1 required positional argument: 'class_id'.Could you tell me the solution?

I am a beginner. I want to use my own dataset. I only change the path of dataset. I already create a new catalog.But I still get the error.
FileNotFoundError: [Errno 2] No such file or directory: './data/imagenet'
How can I solve the problem.

I don't understand why the model is not saved after the total number of steps set. why, after setting the path to save the model, and the total number of steps, at the end of the training I don't see any model saved? What am I missing? Thanks for your attention.

also hardcoded to 2 in dataloader

Did you tried fp16 to increase batch size?

The official BigGAN generator pretrained models were released 2 or 3 weeks ago & can be used on Colab for free evaluations, and made a big splash as people started finding bizarre examples and making things like GANbreeder. Indeed, it's led to a bit of a backlash about it being boring or all-samey: "Helena Sarin: Why Bigger Isn’t Always Better With GANs And AI Art". It's hard to do better, though, since the 512-core TPUv3 slices are not publicly available, and a BigGAN might require 1+ GPU-years (especially with the large minibatches that might be necessary for the highest quality, see issue #1). Another issue is the lack of source code or discriminator pretrained model release, the latter of which is deliberate ("This work was conducted to advance the state of the art in generative adversarial networks for image generation. We are releasing the pre-trained generator to allow our work to be verified, which is standard practice in academia. It does not include the discriminator to minimize the potential for exploitation."), and the former of which seems increasingly unlikely (ajmooch has been asked multiple times and made no promises, suggesting Google/DM isn't going to allow it).

What would be ideal would be the ability to retrain the BigGAN G models. (The lack of discriminator models is a problem but can probably be fixed by setting the G learning rate to 0 and running for a few epochs on ImageNet before re-enabling G. Discriminators seem to learn faster than generators, anyway, so catching up to the G should be relatively quick.) This would let people drop in new datasets for finetuning and get fun new BigGANs which exploit the enormous high-quality prior information embedded in the official generators. It would be a big deal for artists & hobbyists (maybe not so much researchers), especially given that this is an easy-to-use codebase and has support for dropping in a folder-of-folder of images.

I don't know how easy this would be to implement (loading a Tensorflow-made model in PyTorch), but on the bright side, it offers a test of how compatible/accurate a reimplementation this codebase is - if it can't load and retrain the official generators, something's different.

Hello,
can you show me the command for running the code using the pretrained model?

Nice work!!! I’m so glad to find out this PyTorch implementation of the BigGAN work. I would greatly appreciate if you could help me with the below questions :)
(1) Have you been able to reproduce BigGAN on ImageNet?
(2) Do you have any plan to release your PyTorch pre-trained models?
Thanks a lot!

In the BigGAN paper, one of the important features is the use of large minibatches, n=2048 for most of the results. Table 1 shows that FID/Inception improve considerably with, among other things, minibatches going from 256 to 2048 (FID halves, IS almost doubles). In comparison, the other tweaks appear to have smaller benefits.

Obviously, most people don't have a 512-core TPUv3 slice to train on and can't train those minibatches directly. (I'm getting minibatches of 33 on my 2 GPUs right now.) But accumulating gradients across multiple minibatches should be near-identical (at a wallclock multiple slowdown), and might be important for getting the best results. So gradient accumulation should be supported.

As I understand it in PyTorch, gradient accumulation is the default behavior of .backward(), and all that is necessary is to add an inner loop to train.py's train loop around the D/G updates, where d/g_optimizer.step() gets run every i iterations, where i is the total minibatch / actual minibatch. So something like 'for step ... / for i ... / ... / d/g_optimize.step()'.

Which dir are the output results written to? I couldn't find a directory?

I want to run this system for a few hundred iterations just to see that the pipeline works before I run the entire computation. Is that possible? How would I stop after few hundred iterations and get to review the results?

I am training on my custom dataset, which works really good with the default 128 pixels resolution.

I tried setting imsize=256, and using hinge loss.
The generated images are always blank white.
When looking at the program output, the values of the discriminator for both true and generated images are always 0.0 , so obviously something does not work right.

My hypothesis: the fixed count of layers in the generator and discriminator are set for 128 pixels resolution, will adding more conv blocks at some place fix the behaviour for 256 pixels resolution?

thanks a lot for the help

thanks

One useful trick for data cleaning is taking a trained Discriminator and using it to find the 'worst' samples and either manually reviewing them or automatically deleting them.

I modified the training script to support this by ranking a folder dataset using a D model. The default dataloader doesn't preserve image paths, so an additional dataset has to be added:

diff --git a/data_loader.py b/data_loader.py
index 736362c..c4528a8 100755
--- a/data_loader.py
+++ b/data_loader.py
@@ -29,7 +29,7 @@ class Data_Loader():
         transforms = self.transform(True, True, True, False)
         dataset = dsets.LSUN(self.path, classes=classes, transform=transforms)
         return dataset
-    
+^M
     def load_imagenet(self):
         transforms = self.transform(True, True, True, True)
         dataset = dsets.ImageFolder(self.path+'/imagenet', transform=transforms)
@@ -42,9 +42,15 @@ class Data_Loader():
 
     def load_off(self):
         transforms = self.transform(True, True, True, False)
         dataset = dsets.ImageFolder(self.path, transform=transforms)
         return dataset
 
+    def load_rank(self):^M
+        transforms = self.transform(True, True, True, False)^M
+        dataset = ImageFolderWithPaths(self.path, transform=transforms)^M
+        return dataset^M
+^M
     def loader(self):
         if self.dataset == 'lsun':
             dataset = self.load_lsun()
@@ -54,6 +60,8 @@ class Data_Loader():
             dataset = self.load_celeb()
         elif self.dataset == 'off':
             dataset = self.load_off()
+        elif self.dataset == 'rank':^M
+            dataset = self.load_rank()^M
 
         print('dataset',len(dataset))
         loader = torch.utils.data.DataLoader(dataset=dataset,
@@ -63,3 +71,18 @@ class Data_Loader():
                                               drop_last=True)
         return loader
 
+^M
+class ImageFolderWithPaths(dsets.ImageFolder):^M
+    """Custom dataset that includes image file paths. Extends^M
+    torchvision.datasets.ImageFolder^M
+    """^M
+^M
+    # override the __getitem__ method. this is the method dataloader calls^M
+    def __getitem__(self, index):^M
+        # this is what ImageFolder normally returns^M
+        original_tuple = super(ImageFolderWithPaths, self).__getitem__(index)^M
+        # the image file path^M
+        path = self.imgs[index][0]^M
+        # make a new tuple that includes original and the path^M
+        tuple_with_path = (original_tuple + (path,))^M
+        return tuple_with_path^M
diff --git a/parameter.py b/parameter.py
index 0b59c5a..1191208 100755
--- a/parameter.py
+++ b/parameter.py
@@ -37,7 +37,7 @@ def get_parameters():
     parser.add_argument('--train', type=str2bool, default=True)
     parser.add_argument('--parallel', type=str2bool, default=False)
     parser.add_argument('--gpus', type=str, default='0', help='gpuids eg: 0,1,2,3  --parallel True  ')
-    parser.add_argument('--dataset', type=str, default='lsun', choices=['lsun', 'celeb','off'])
+    parser.add_argument('--dataset', type=str, default='lsun', choices=['lsun', 'celeb','off', 'rank'])^M
     parser.add_argument('--use_tensorboard', type=str2bool, default=False)
 
     # Path

Then a ranker.py script can be based on train.py:

import torch
import torch.nn as nn
from model_resnet import Discriminator
# from utils import *
from parameter import *
from data_loader import Data_Loader
from torch.backends import cudnn
import os

class Trainer(object):
    def __init__(self, data_loader, config):

        # Data loaders
        self.data_loader = data_loader

        # exact model and loss
        self.model = config.model

        # Model hyper-parameters
        self.imsize = config.imsize
        self.parallel = config.parallel
        self.gpus = config.gpus

        self.batch_size = config.batch_size
        self.num_workers = config.num_workers
        self.pretrained_model = config.pretrained_model

        self.dataset = config.dataset
        self.image_path = config.image_path
        self.version = config.version

        self.n_class = 1000 # config.n_class TODO
        self.chn = config.chn

        # Path
        self.model_save_path = os.path.join(config.model_save_path, self.version)

        self.device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
        self.build_model()

        # Start with trained model
        self.load_pretrained_model()
        self.train()

    def train(self):

        self.D.train()
        # Data iterator
        data_iter = iter(self.data_loader)

        total_steps = self.data_loader.__len__()
        for step in range(0, total_steps):
            real_images, real_labels, real_paths = next(data_iter)
            real_labels = real_labels.to(self.device)
            real_images = real_images.to(self.device)
            d_out_real = self.D(real_images, real_labels)

            rankings = d_out_real.data.tolist()
            for i in range(0, len(real_paths)):
                print(real_paths[i], rankings[i])


    def load_pretrained_model(self):
        self.D.load_state_dict(torch.load(os.path.join(
            self.model_save_path, '{}_D.pth'.format(self.pretrained_model))))

    def build_model(self):
        # code_dim=100, n_class=1000
        self.D = Discriminator(self.n_class, chn=self.chn).to(self.device)
        if self.parallel:
            gpus = [int(i) for i in self.gpus.split(',')]
            self.D = nn.DataParallel(self.D, device_ids=gpus)

def main(config):
    # For fast training
    cudnn.benchmark = True

    # Data loader
    data_loader = Data_Loader(config.train, config.dataset, config.image_path, config.imsize,
                              config.batch_size, shuf=False)
    Trainer(data_loader.loader(), config)

if __name__ == '__main__':
    config = get_parameters()
    # print(config)
    main(config)

Example use:

$ python ranker.py --batch_size 1  --dataset rank --image_path /media/gwern/Data/danbooru2017/characters-faces/ --version 1kfaces --parallel True --gpus 0,1 --pretrained 627003
dataset 750356
/media/gwern/Data/danbooru2017/characters-faces/2k-tan/10139.jpg0.png 3.864508628845215
/media/gwern/Data/danbooru2017/characters-faces/2k-tan/104024.jpg0.png 3.384716272354126
/media/gwern/Data/danbooru2017/characters-faces/2k-tan/104024.jpg1.png 3.004866600036621
/media/gwern/Data/danbooru2017/characters-faces/2k-tan/1054281.jpg0.png 3.4808170795440674
/media/gwern/Data/danbooru2017/characters-faces/2k-tan/108431.jpg0.png 3.6894052028656006
/media/gwern/Data/danbooru2017/characters-faces/2k-tan/108805.jpg0.png 3.898812770843506
/media/gwern/Data/danbooru2017/characters-faces/2k-tan/111774.jpg0.png 2.8409836292266846
/media/gwern/Data/danbooru2017/characters-faces/2k-tan/114465.jpg0.png 3.883681058883667
/media/gwern/Data/danbooru2017/characters-faces/2k-tan/115171.jpg0.png 4.3082122802734375
...

Some cleaner built-in support would be good.

Traceback (most recent call last):
File "main.py", line 3, in
from trainer import Trainer
File "/home/user/shiyu/BigGAN-pytorch/trainer.py", line 11, in
from model_resnet import Generator, Discriminator
File "/home/user/shiyu/BigGAN-pytorch/model_resnet.py", line 23
v = weight_mat.t() @ u
^
SyntaxError: invalid syntax

I have this error. Is my pytorch version too old?

I used the suggested scripts to train on ImageNet and LSUN(2 class). But cannot get the reported results. Most generated images collapse to the same pattern.

https://github.com/AaronLeong/BigGAN-pytorch/blob/master/model_resnet.py#L161

It seems that bias=True no matter if bn is True or False.