yiranran / unpaired-portrait-drawing Goto Github PK

Code for Unpaired Portrait Drawing Generation via Asymmetric Cycle Mapping (CVPR 2020)

Python 99.02% MATLAB 0.80% Shell 0.17%

unpaired-portrait-drawing's Introduction

Unpaired Portrait Drawing Generation via Asymmetric Cycle Mapping

We provide PyTorch implementations for our CVPR 2020 paper "Unpaired Portrait Drawing Generation via Asymmetric Cycle Mapping". paper, suppl.

This project generates multi-style artistic portrait drawings from face photos using a GAN-based model.

[Jittor implementation]

Our Proposed Framework

Sample Results

From left to right: input, output(style1), output(style2), output(style3)

Prerequisites

Linux or macOS
Python 3
CPU or NVIDIA GPU + CUDA CuDNN

Installation

To install the dependencies, run

pip install -r requirements.txt

Colab

A colab demo is here.

Test steps (apply a pretrained model)

1. Download pre-trained models from BaiduYun(extract code:c9h7) or GoogleDrive and rename the folder to checkpoints.
1. Test for example photos: generate artistic portrait drawings for example photos in the folder ./examples using

# with GPU
python test_seq_style.py
# without GPU
python test_seq_style.py --gpu -1

The test results will be saved to a html file here: ./results/pretrained/test_200/index3styles.html. The result images are saved in ./results/pretrained/test_200/images3styles, where real, fake1, fake2, fake3 correspond to input face photo, style1 drawing, style2 drawing, style3 drawing respectively.

1. To test on your own photos: First use an image editor to crop the face region of your photo (or use an optional preprocess here). Then specify the folder that contains test photos using option --dataroot, specify save folder name using option --savefolder and run the above command again:

# with GPU
python test_seq_style.py --dataroot [input_folder] --savefolder [save_folder_name]
# without GPU
python test_seq_style.py --gpu -1 --dataroot [input_folder] --savefolder [save_folder_name]
# E.g.
python test_seq_style.py --gpu -1 --dataroot ./imgs/test1 --savefolder 3styles_test1

The test results will be saved to a html file here: ./results/pretrained/test_200/index[save_folder_name].html. The result images are saved in ./results/pretrained/test_200/images[save_folder_name]. An example html screenshot is shown below:

You can contact email [email protected] for any questions.

Train steps

1. Prepare for the dataset: 1) download face photos and portrait drawings from internet (e.g. resources). 2) align, crop photos and drawings & 3) prepare nose, eyes, lips masks according to preprocess instructions. 3) put aligned photos under ./datasets/portrait_drawing/train/A, aligned drawings under ./datasets/portrait_drawing/train/B, masks under A_nose,A_eyes,A_lips,B_nose,B_eyes,B_lips respectively.
1. Train a 3-class style classifier and extract the 3-dim style feature (according to paper). And save the style feature of each drawing in the training set in .npy format, in folder ./datasets/portrait_drawing/train/B_feat

A subset of our training set is here.

1. Train our model

sh ./scripts/train.sh

Models are saved in folder checkpoints/portrait_drawing

Citation

If you use this code for your research, please cite our paper.

@inproceedings{YiLLR20,
  title     = {Unpaired Portrait Drawing Generation via Asymmetric Cycle Mapping},
  author    = {Yi, Ran and Liu, Yong-Jin and Lai, Yu-Kun and Rosin, Paul L},
  booktitle = {{IEEE} Conference on Computer Vision and Pattern Recognition (CVPR '20)},
  pages     = {8214--8222},
  year      = {2020}
}

Acknowledgments

Our code is inspired by pytorch-CycleGAN-and-pix2pix.

unpaired-portrait-drawing's People

Contributors

Stargazers

Watchers

unpaired-portrait-drawing's Issues

About preprocessing

I need your help!

Who can tell me how to use the face parsing network in https://github.com/cientgu/Mask_Guided_Portrait_Editing?

Looking forward to reply!

link of paper

Hi， Would give a link of paper in ReadMe?

pretrained_networks.py is unavailable

from . import pretrained_networks as pn

but pretrained_networks not find

training error

hi I'm trying to train with your provided subset and here is the error:

Original Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop
data = fetcher.fetch(index)
File "/opt/conda/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 51, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/opt/conda/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 51, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/jovyan/Unpaired-Portrait-Drawing-master/data/unaligned_mask_stylecls_dataset.py", line 45, in getitem
A_mask_img = Image.open(os.path.join(self.auxdir_A+'_nose',basenA))
File "/opt/conda/lib/python3.10/site-packages/PIL/Image.py", line 3243, in open
fp = builtins.open(filename, "rb")
FileNotFoundError: [Errno 2] No such file or directory: './portrait_drawing_subset/train/A_nose/inthewild-ia_portrait_10033_resized-checkpoint.png'
what shuld I do?
thank you!