cientgu / mask_guided_portrait_editing

pytorch implementation of "Mask-Guided Portrait Editing with Conditional GANs"

Mask-Guided Portrait Editing with Conditional GANs

This is an official pytorch implementation of "Mask-Guided Portrait Editing with Conditional GANs"(CVPR2019). The major contributors of this repository include Shuyang Gu, Jianmin Bao, Hao Yang, Dong Chen, Fang Wen, Lu Yuan at Microsoft Research.

Introduction

Mask-Guided Portrait Editing is a novel technology based on mask-guided condititonal GANs, which can synthesize diverse, high-quality and controllable facial images from given masks. With the changeable input facial mask and source image, this method allows users to do high-level portrait editing.

Citation

If you find our code helpful for your research, please consider citing:

@inproceedings{gu2019mask,
  title={Mask-Guided Portrait Editing With Conditional GANs},
  author={Gu, Shuyang and Bao, Jianmin and Yang, Hao and Chen, Dong and Wen, Fang and Yuan, Lu},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  pages={3436--3445},
  year={2019}
} 

Getting Started

Prerequisite

• Linux.
• Pytorch 0.4.1.
• Nvidia GPU: K40, M40, P100.
• CUDA9.2 or 10.

Running code

• download pretrained models here, put it under folder checkpoints/pretrained .
• component editing: ./scripts/test_edit.sh
• component transfer: ./scripts/test_edit_free_encode.sh change the corresponding component file in results/pretrained/editfree_latest, then run: ./scripts/test_edit_free_generate.sh get the component transfer results.
• training: ./scripts/train.sh

mask_guided_portrait_editing's Issues

Supplementary Material

Hi,

In the paper some details are deferred to the "supplementary material", is this available online?

Thanks

Could you release the face parsing model PF?

I wanted to generate masks of some samples,but didn't see the PF model. Thanks

training data

Hello, the training code cannot be trained due to lack of training data (train_img and train_label), can you send me the complete training data?
My email is [email protected]. Thank you.

Generate mask for custom images?

How do I generate mask(label) for a custom image? As of now, the masks are already given here for the example images. Thank you.

test results are weird

Hi,

When I test the downloaded models, the generated results are weird:

The reconstructed style and content images are just some random noise. Thank you very much!

Local VAE

Hi! Author!

I have a question about local embedding encoder!

1. Why did you use VAE in train code, not auto encoder (like paper) ?
   There inserted decoder feature, not encoder feature in the 3D zero tensor...

At the local phase, auto encoder seems to be more appropriate because feature compression is performed.

2. Can you tell me how the size criteria for the components of face are set? (32 x 48 , 80 x 144 ..)

Thank you for amazing contribution!

./datasets/helen_align/train_label is not a valid directory

------------ Options -------------
batchSize: 4
beta1: 0.5
checkpoints_dir: ./checkpoints
continue_train: False
cross_embed: False
data_augmentation: True
data_type: 32
dataroot: ./datasets/helen_align/
debug: False
debug_mask_part: True
display_freq: 30
display_winsize: 512
dist_model: False
embed_length: 128
embed_nc: 64
face_rec_model:
feat_num: 3
fineSize: 512
gpu_ids: [0, 1]
input_nc: 3
instance_feat: False
isTrain: True
label_feat: False
label_nc: 11
lambda_feat: 10.0
loadSize: 1024
load_features: False
load_pretrain:
longSize: 256
lr: 0.0002
mask_model: True
max_dataset_size: inf
model: pix2pixHD
nThreads: 2
n_blocks_global: 9
n_blocks_local: 3
n_clusters: 10
n_downsample_E: 4
n_downsample_global: 2
n_layers_D: 3
n_local_enhancers: 1
name: face_256
ndf: 64
nef: 16
netG: global
ngf: 64
niter: 10
niter_decay: 45
niter_fix_global: 0
no_flip: False
no_ganFeat_loss: False
no_grammatrix_loss: False
no_html: False
no_instance: True
no_l2_loss: False
no_lsgan: True
no_vgg_loss: False
norm: batch
num_D: 2
output_nc: 3
phase: train
pool_size: 0
print_freq: 100
resize_or_crop: none
sample_noise: False
save_epoch_freq: 10
save_latest_freq: 15000
serial_batches: False
tf_log: False
use_dropout: False
use_parsing_label: False
vae_encoder: True
verbose: True
which_epoch: latest
-------------- End ----------------
CustomDatasetDataLoader
dataset [AlignedDataset] was created
Traceback (most recent call last):
File "train.py", line 39, in
data_loader = CreateDataLoader(opt)
File "/content/Mask_Guided_Portrait_Editing/data/data_loader.py", line 6, in CreateDataLoader
data_loader.initialize(opt)
File "/content/Mask_Guided_Portrait_Editing/data/custom_dataset_data_loader.py", line 20, in initialize
self.dataset = CreateDataset(opt)
File "/content/Mask_Guided_Portrait_Editing/data/custom_dataset_data_loader.py", line 11, in CreateDataset
dataset.initialize(opt)
File "/content/Mask_Guided_Portrait_Editing/data/aligned_dataset.py", line 23, in initialize
self.A_paths = sorted(make_dataset(self.dir_A))
File "/content/Mask_Guided_Portrait_Editing/data/image_folder.py", line 23, in make_dataset
assert os.path.isdir(dir), '%s is not a valid directory' % dir
AssertionError: ./datasets/helen_align/train_label is not a valid directory

How to edit mask?

Hello~ Thank you for sharing such nice project. I did not find a easy find to modify mask. Would you mind plz provide mask editor if possible? Thank u in advance