zpdesu / barbershop

Barbershop: GAN-based Image Compositing using Segmentation Masks (SIGGRAPH Asia 2021)

Barbershop: GAN-based Image Compositing using Segmentation Masks

Barbershop: GAN-based Image Compositing using Segmentation Masks
Peihao Zhu, Rameen Abdal, John Femiani, Peter Wonka

arXiv | BibTeX | Project Page | Video

Abstract Seamlessly blending features from multiple images is extremely challenging because of complex relationships in lighting, geometry, and partial occlusion which cause coupling between different parts of the image. Even though recent work on GANs enables synthesis of realistic hair or faces, it remains difficult to combine them into a single, coherent, and plausible image rather than a disjointed set of image patches. We present a novel solution to image blending, particularly for the problem of hairstyle transfer, based on GAN-inversion. We propose a novel latent space for image blending which is better at preserving detail and encoding spatial information, and propose a new GAN-embedding algorithm which is able to slightly modify images to conform to a common segmentation mask. Our novel representation enables the transfer of the visual properties from multiple reference images including specific details such as moles and wrinkles, and because we do image blending in a latent-space we are able to synthesize images that are coherent. Our approach avoids blending artifacts present in other approaches and finds a globally consistent image. Our results demonstrate a significant improvement over the current state of the art in a user study, with users preferring our blending solution over 95 percent of the time.

Description

Official Implementation of Barbershop. KEEP UPDATING! Please Git Pull the latest version.

Updates

2021/12/27 Add dilation and erosion parameters to smooth the boundary.

2021/12/24 Important Update: Add semantic mask inpainting module to solve the occlusion problem. Please git pull the latest version.

2021/12/18 Add a rough version of the project.

2021/06/02 Add project page.

Installation

• Clone the repository:

git clone https://github.com/ZPdesu/Barbershop.git
cd Barbershop

• Dependencies:
  We recommend running this repository using Anaconda. All dependencies for defining the environment are provided in environment/environment.yaml.

Download II2S images

Please download the II2S and put them in the input/face folder.

Getting Started

Preprocess your own images. Please put the raw images in the unprocessed folder.

python align_face.py

Produce realistic results:

python main.py --im_path1 90.png --im_path2 15.png --im_path3 117.png --sign realistic --smooth 5

Produce results faithful to the masks:

python main.py --im_path1 90.png --im_path2 15.png --im_path3 117.png --sign fidelity --smooth 5

Todo List

• add a detailed readme
• update mask inpainting code
• integrate image encoder
• add preprocessing step
• ...

Acknowledgments

This code borrows heavily from II2S.

BibTeX

@misc{zhu2021barbershop,
      title={Barbershop: GAN-based Image Compositing using Segmentation Masks},
      author={Peihao Zhu and Rameen Abdal and John Femiani and Peter Wonka},
      year={2021},
      eprint={2106.01505},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

barbershop's Issues

Model does not correctly model changes to facial hair.

I tested an image of myself, cleanshaven, and supplied it with the image of another man with a large beard and facing approximately the same direction. However, while the AI did correctly interpolate the hair atop my head, it did not supply me with the other man's beard.

ImportError: No module named 'fused'

when i run the command:
python main.py --im_path1 90.png --im_path2 15.png --im_path3 117.png --sign realistic --smooth 5

i meet the following problem:

Traceback (most recent call last):
File "main.py", line 13, in
from models.Embedding import Embedding
File "/home/yaoyuntao/programs/Barbershop-main/models/Embedding.py", line 3, in
from models.Net import Net
File "/home/yaoyuntao/programs/Barbershop-main/models/Net.py", line 3, in
from models.stylegan2.model import Generator
File "/home/yaoyuntao/programs/Barbershop-main/models/stylegan2/model.py", line 11, in
from models.stylegan2.op import FusedLeakyReLU, fused_leaky_relu, upfirdn2d
File "/home/yaoyuntao/programs/Barbershop-main/models/stylegan2/op/init.py", line 1, in
from .fused_act import FusedLeakyReLU, fused_leaky_relu
File "/home/yaoyuntao/programs/Barbershop-main/models/stylegan2/op/fused_act.py", line 14, in
os.path.join(module_path, "fused_bias_act_kernel.cu"),
File "/home/yaoyuntao/anaconda3/envs/Barbershop/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 997, in load
keep_intermediates=keep_intermediates)
File "/home/yaoyuntao/anaconda3/envs/Barbershop/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1213, in _jit_compile
return _import_module_from_library(name, build_directory, is_python_module)
File "/home/yaoyuntao/anaconda3/envs/Barbershop/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1560, in _import_module_from_library
file, path, description = imp.find_module(module_name, [path])
File "/home/yaoyuntao/anaconda3/envs/Barbershop/lib/python3.7/imp.py", line 296, in find_module
raise ImportError(_ERR_MSG.format(name), name=name)
ImportError: No module named 'fused'

who can tell me the reason? thanks

OSError: /apply/anaconda3/envs/Barber_shop/lib/python3.7/site-packages/torch/lib/../../../../libcublas.so.11: undefined symbol: free_gemm_select, version libcublasLt.so.11

python main.py --im_path1 1.png --im_path2 2.png --im_path3 3.png --sign realistic --smooth 7
Traceback (most recent call last):
File "main.py", line 3, in
import torch
File "/apply/anaconda3/envs/Barber_shop/lib/python3.7/site-packages/torch/init.py", line 189, in
_load_global_deps()
File "/apply/anaconda3/envs/Barber_shop/lib/python3.7/site-packages/torch/init.py", line 142, in _load_global_deps
ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
File "/apply/anaconda3/envs/Barber_shop/lib/python3.7/ctypes/init.py", line 364, in init
self._handle = _dlopen(self._name, mode)
OSError: /apply/anaconda3/envs/Barber_shop/lib/python3.7/site-packages/torch/lib/../../../../libcublas.so.11: undefined symbol: free_gemm_select, version libcublasLt.so.11

pickle error

Great work!

BiSeNet is not creating the correct mask

I have trained BiSeNet for my custom dataset and the individual BiSeNet notebook seems to be giving correct results for my data, however when I use the same weights in Barbershop it does not give me the correct target segmentation mask. I have changed seg_mean and seg_std, according to my dataset, but the target segmentation mask is not correctly generated. Could you please guide me regarding what other changes am I supposed to make for custom dataset.

Problem for PCA.npz file

Hi, thanks for this amazing job!

When I remove PCA.npz for related pt, I find the result don't have big change.

So I wanna know the usage and necessary of PCA.npz file, and How can I generate it for my own model?
Thanks

licence

Hi! could you pls publish LICENCE file with terms of usage your code?

re-train stylegan2

We found that the your stylegan2 network parameter name is inconsistent with the original stylegan2. If we change the network structure of the original Stylegan2 to be the same as yours, and the other functions remain the same as the original Stylegan2. Is it possible to map the trained parameters as the inference· parameters of your Stylegan2 network? thanks for your reply

RuntimeError: std::bad_alloc

When i run python align_face.py, error will happen:

911.jpg: Number of faces detected: 1
Traceback (most recent call last):
File "/data/juicefs_hz_cv_v3/11146533/Barbershop/align_face.py", line 42, in
face_tensor = torchvision.transforms.ToTensor()(face).unsqueeze(0).cuda()
File "/root/anaconda3/lib/python3.9/site-packages/torchvision/transforms/transforms.py", line 104, in call
return F.to_tensor(pic)
File "/root/anaconda3/lib/python3.9/site-packages/torchvision/transforms/functional.py", line 96, in to_tensor
img = torch.ByteTensor(torch.ByteStorage.from_buffer(pic.tobytes()))
RuntimeError: std::bad_alloc

Anyone help me please!

Can you make a dockerfile for this?

I'm trying to build a docker file using pytorch/pytorch, but it gets stuck in cuda libraries, I think anaconda has different cuda and cudatoolkit uses different

ninja: build stopped: subcommand failed.

Thank you for your great work here? When I run main.py I got this error.

RuntimeError: Error building extension 'fused': [1/2] /usr/local/cuda-11.2/bin/nvcc -DTORCH_EXTENSION_NAME=fused -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem /home/lhf/data/[/home/lhf/data/anaconda3]/envs/bar/lib/python3.8/site-packages/torch/include -isystem /home/lhf/data/[/home/lhf/data/anaconda3]/envs/bar/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /home/lhf/data/[/home/lhf/data/anaconda3]/envs/bar/lib/python3.8/site-packages/torch/include/TH -isystem /home/lhf/data/[/home/lhf/data/anaconda3]/envs/bar/lib/python3.8/site-packages/torch/include/THC -isystem /usr/local/cuda-11.2/include -isystem /home/lhf/data/[/home/lhf/data/anaconda3]/envs/bar/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -std=c++14 -c /media/pc/nvme1n1/lhf/Barbershop/models/stylegan2/op/fused_bias_act_kernel.cu -o fused_bias_act_kernel.cuda.o
FAILED: fused_bias_act_kernel.cuda.o
/usr/local/cuda-11.2/bin/nvcc -DTORCH_EXTENSION_NAME=fused -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem /home/lhf/data/[/home/lhf/data/anaconda3]/envs/bar/lib/python3.8/site-packages/torch/include -isystem /home/lhf/data/[/home/lhf/data/anaconda3]/envs/bar/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /home/lhf/data/[/home/lhf/data/anaconda3]/envs/bar/lib/python3.8/site-packages/torch/include/TH -isystem /home/lhf/data/[/home/lhf/data/anaconda3]/envs/bar/lib/python3.8/site-packages/torch/include/THC -isystem /usr/local/cuda-11.2/include -isystem /home/lhf/data/[/home/lhf/data/anaconda3]/envs/bar/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -std=c++14 -c /media/pc/nvme1n1/lhf/Barbershop/models/stylegan2/op/fused_bias_act_kernel.cu -o fused_bias_act_kernel.cuda.o
/media/pc/nvme1n1/lhf/Barbershop/models/stylegan2/op/fused_bias_act_kernel.cu:7:10: fatal error: torch/types.h: 没有那个文件或目录
#include <torch/types.h>
^~~~~~~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.

Will the code of the Face swapping experiment part of the paper be made public?

The function like Fig. 7 in paper will be public?

ninja: build stopped: subcommand failed.

COMMAND
python3 main.py

ENVS
ubuntu 18.04

NVIDIA-SMI 470.86 Driver Version: 470.86 CUDA Version: 11.4
GeForce RTX 3090

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Wed_Jul_22_19:09:09_PDT_2020
Cuda compilation tools, release 11.0, V11.0.221
Build cuda_11.0_bu.TC445_37.28845127_0

ERROR

Traceback (most recent call last):
File "/opt/conda/envs/barber/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1539, in _run_ninja_build
env=env)
File "/opt/conda/envs/barber/lib/python3.7/subprocess.py", line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "main.py", line 13, in
from models.Embedding import Embedding
File "/home/user/jupyter-work/barber/models/Embedding.py", line 3, in
from models.Net import Net
File "/home/user/jupyter-work/barber/models/Net.py", line 3, in
from models.stylegan2.model import Generator
File "/home/user/jupyter-work/barber/models/stylegan2/model.py", line 11, in
from models.stylegan2.op import FusedLeakyReLU, fused_leaky_relu, upfirdn2d
File "/home/user/jupyter-work/barber/models/stylegan2/op/init.py", line 1, in
from .fused_act import FusedLeakyReLU, fused_leaky_relu
File "/home/user/jupyter-work/barber/models/stylegan2/op/fused_act.py", line 17, in
os.path.join(module_path, "fused_bias_act_kernel.cu"),
File "/opt/conda/envs/barber/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 997, in load
keep_intermediates=keep_intermediates)
File "/opt/conda/envs/barber/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1202, in jit_compile
with_cuda=with_cuda)
File "/opt/conda/envs/barber/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1300, in write_ninja_file_and_build_library
error_prefix="Error building extension '{}'".format(name))
File "/opt/conda/envs/barber/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1555, in run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error building extension 'fused': [1/3] /usr/local/cuda-11.0/bin/nvcc -DTORCH_EXTENSION_NAME=fused -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include -isystem /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/TH -isystem /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/THC -isystem /usr/local/cuda-11.0/include -isystem /opt/conda/envs/barber/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -std=c++14 -c /home/user/jupyter-work/barber/models/stylegan2/op/fused_bias_act_kernel.cu -o fused_bias_act_kernel.cuda.o
FAILED: fused_bias_act_kernel.cuda.o
/usr/local/cuda-11.0/bin/nvcc -DTORCH_EXTENSION_NAME=fused -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include -isystem /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/TH -isystem /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/THC -isystem /usr/local/cuda-11.0/include -isystem /opt/conda/envs/barber/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -std=c++14 -c /home/user/jupyter-work/barber/models/stylegan2/op/fused_bias_act_kernel.cu -o fused_bias_act_kernel.cuda.o
nvcc fatal : Unsupported gpu architecture 'compute_86'
[2/3] c++ -MMD -MF fused_bias_act.o.d -DTORCH_EXTENSION_NAME=fused -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -isystem /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include -isystem /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/TH -isystem /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/THC -isystem /usr/local/cuda-11.0/include -isystem /opt/conda/envs/barber/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -c /home/user/jupyter-work/barber/models/stylegan2/op/fused_bias_act.cpp -o fused_bias_act.o
In file included from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/c10/core/DeviceType.h:8:0,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/c10/core/Device.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/c10/core/Allocator.h:6,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/ATen/ATen.h:7,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/extension.h:4,
from /home/user/jupyter-work/barber/models/stylegan2/op/fused_bias_act.cpp:1:
/home/user/jupyter-work/barber/models/stylegan2/op/fused_bias_act.cpp: In function ‘at::Tensor fused_bias_act(const at::Tensor&, const at::Tensor&, const at::Tensor&, int, int, float, float)’:
/home/user/jupyter-work/barber/models/stylegan2/op/fused_bias_act.cpp:7:42: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
#define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
^
/home/user/jupyter-work/barber/models/stylegan2/op/fused_bias_act.cpp:13:5: note: in expansion of macro ‘CHECK_CUDA’
CHECK_CUDA(input);
^~~~~~~~~~
In file included from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/ATen/Tensor.h:3:0,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/ATen/Context.h:4,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/ATen/ATen.h:9,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/extension.h:4,
from /home/user/jupyter-work/barber/models/stylegan2/op/fused_bias_act.cpp:1:
/opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/ATen/core/TensorBody.h:277:30: note: declared here
DeprecatedTypeProperties & type() const {
^~~~
In file included from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/c10/core/DeviceType.h:8:0,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/c10/core/Device.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/c10/core/Allocator.h:6,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/ATen/ATen.h:7,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/extension.h:4,
from /home/user/jupyter-work/barber/models/stylegan2/op/fused_bias_act.cpp:1:
/home/user/jupyter-work/barber/models/stylegan2/op/fused_bias_act.cpp:7:42: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
#define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
^
/home/user/jupyter-work/barber/models/stylegan2/op/fused_bias_act.cpp:14:5: note: in expansion of macro ‘CHECK_CUDA’
CHECK_CUDA(bias);
^~~~~~~~~~
In file included from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/ATen/Tensor.h:3:0,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/ATen/Context.h:4,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/ATen/ATen.h:9,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader_options.h:4,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/base.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader/stateful.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data/dataloader.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/data.h:3,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:8,
from /opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/torch/extension.h:4,
from /home/user/jupyter-work/barber/models/stylegan2/op/fused_bias_act.cpp:1:
/opt/conda/envs/barber/lib/python3.7/site-packages/torch/include/ATen/core/TensorBody.h:277:30: note: declared here
DeprecatedTypeProperties & type() const {
^~~~
ninja: build stopped: subcommand failed.

Image Encoder

When do you expect to release the Image encoder ?

how to optimize the inference time

Hello, ZPdesu,thank you for your impressive work.I want save the proprecess informations file(.npy) of Image3 and Imge3 before transfer their hair structure and appeance to the new image1,can this idea realise?

Google Colab

I'm excited to watch the demo videos, especially the part of faceswap.
Please release the code for Google colab so that we can try this repo easily.
I am looking forward to your next update.

Cuda out of memory

The first time it worked well on my computer, but the second time I got this error.

RuntimeError: CUDA out of memory. Tried to allocate 128.00 MiB (GPU 0; 2.95 GiB total capacity; 1.86 GiB already allocated; 112.31 MiB free; 1.88 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

I tried the solutions below, and still facing the same issue.
Solution 1: add torch.cuda.empty_cache() at the beginning of main.py
Solution 2: reduce batch_size to 1 in /models/face_parsing/modules)/functions.py

Anyone has an idea?

lots of errors

❯ python main.py --im_path1 90.png --im_path2 15.png --im_path3 117.png --sign realistic --smooth 5
Loading StyleGAN2 from checkpoint: pretrained_models/ffhq.pt
Setting up Perceptual loss...
Traceback (most recent call last):
  File "main.py", line 117, in <module>
    main(args)
  File "main.py", line 18, in main
    ii2s = Embedding(args)
  File "/home/fg/dev/python/Barbershop/models/Embedding.py", line 25, in __init__
    self.setup_embedding_loss_builder()
  File "/home/fg/dev/python/Barbershop/models/Embedding.py", line 94, in setup_embedding_loss_builder
    self.loss_builder = EmbeddingLossBuilder(self.opts)
  File "/home/fg/dev/python/Barbershop/losses/embedding_loss.py", line 18, in __init__
    self.percept = lpips.PerceptualLoss(model="net-lin", net="vgg", use_gpu=use_gpu)
  File "/home/fg/dev/python/Barbershop/losses/lpips/__init__.py", line 22, in __init__
    self.model.initialize(model=model, net=net, use_gpu=use_gpu, colorspace=colorspace, spatial=self.spatial, gpu_ids=gpu_ids)
  File "/home/fg/dev/python/Barbershop/losses/lpips/dist_model.py", line 63, in initialize
    use_dropout=True, spatial=spatial, version=version, lpips=True)
  File "/home/fg/dev/python/Barbershop/losses/lpips/networks_basic.py", line 50, in __init__
    self.net = net_type(pretrained=not self.pnet_rand, requires_grad=self.pnet_tune)
  File "/home/fg/dev/python/Barbershop/losses/lpips/pretrained_networks.py", line 100, in __init__
    vgg_pretrained_features = tv.vgg16(pretrained=pretrained).features
  File "/home/fg/anaconda3/envs/plswork/lib/python3.7/site-packages/torchvision/models/vgg.py", line 150, in vgg16
    return _vgg('vgg16', 'D', False, pretrained, progress, **kwargs)
  File "/home/fg/anaconda3/envs/plswork/lib/python3.7/site-packages/torchvision/models/vgg.py", line 93, in _vgg
    progress=progress)
  File "/home/fg/anaconda3/envs/plswork/lib/python3.7/site-packages/torch/hub.py", line 559, in load_state_dict_from_url
    return torch.load(cached_file, map_location=map_location)
  File "/home/fg/anaconda3/envs/plswork/lib/python3.7/site-packages/torch/serialization.py", line 595, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/home/fg/anaconda3/envs/plswork/lib/python3.7/site-packages/torch/serialization.py", line 781, in _legacy_load
    deserialized_objects[key]._set_from_file(f, offset, f_should_read_directly)
RuntimeError: unexpected EOF, expected 477241 more bytes. The file might be corrupted.

the time to release the code

Thanks for your work! when will the code be released?

Always look forward to the author's update.

Have deep expectations for hairstyles, come on!

Black images output

Hi

I had run the model with provided images and with a couple of my images, but the output is always simply black images.
There are no critical errors in the script output, warnings only.
How do you think the warning is root cause of the error?

~/Barbershop$ python main.py --im_path1 90.png --im_path2 15.png --im_path3 117.png
Loading StyleGAN2 from checkpoint: pretrained_models/ffhq.pt
Setting up Perceptual loss...
Loading model from: /home/pawa/Barbershop/losses/lpips/weights/v0.1/vgg.pth
...[net-lin [vgg]] initialized
...Done
Number of images: 3
Images: 100%|██| 3/3 [02:57<00:00, 59.25s/it]
Number of images: 3
Images: 100%|████| 3/3 [00:35<00:00, 11.89s/it]
Loading StyleGAN2 from checkpoint: pretrained_models/ffhq.pt
Align Step 2:   0%|                                                                                                                                                                                                                                                                                  | 0/100 [00:00<?, ?it/s]/home/pawa/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py:3680: UserWarning: The default behavior for interpolate/upsample with float scale_factor changed in 1.6.0 to align with other frameworks/libraries, and now uses scale_factor directly, instead of relying on the computed output size. If you wish to restore the old behavior, please set recompute_scale_factor=True. See the documentation of nn.Upsample for details.
  "The default behavior for interpolate/upsample with float scale_factor changed "
/home/pawa/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py:3635: UserWarning: Default upsampling behavior when mode=bicubic is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  "See the documentation of nn.Upsample for details.".format(mode)
Loading StyleGAN2 from checkpoint: pretrained_models/ffhq.pt
Setting up Perceptual loss...
Loading model from: /home/pawa/Barbershop/losses/masked_lpips/weights/v0.1/vgg.pth
...[net-lin [vgg]] initialized
...Done
Setting up Perceptual loss...
Loading model from: /home/pawa/Barbershop/losses/masked_lpips/weights/v0.1/vgg.pth
...[net-lin [vgg]] initialized
...Done

RuntimeError: Found no NVIDIA driver on your system

I get this output when running python align_face.py:

Downloading Shape Predictor
Downloading https://drive.google.com/uc?id=1huhv8PYpNNKbGCLOaYUjOgR1pY5pmbJx ... done
photo_2022-05-01_17-32-49.jpg: Number of faces detected: 1
Traceback (most recent call last):
  File "align_face.py", line 42, in <module>
    face_tensor = torchvision.transforms.ToTensor()(face).unsqueeze(0).cuda()
  File "~/anaconda3/envs/Barbershop/lib/python3.7/site-packages/torch/cuda/__init__.py", line 172, in _lazy_init
    torch._C._cuda_init()
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx

Can you please specify in readme, that machine with nvidia card is required for this project to run?

Little question about cal_loss

Hi,
Thank you for this amazing works. I have a little question about this funtion at this line

def cal_loss(self, im_dict, latent_in, latent_F=None, F_init=None):

In both "invert_images_in_FS" & "invert_images_in_w", it seems that you didn't pass "latent_F" & "F_init" in cal_loss to do the computation below:

   if latent_F is not None and F_init is not None:
      l_F = self.net.cal_l_F(latent_F, F_init)
       loss_dic['l_F'] = l_F
       loss += l_F

I wonder that should we still calaulate l_F loss in somewhere? or I misunderstanding somethings?

BR,
Ziv

about BiSeNet ckpt

Hi, I would like to know what dataset is the BiSeNet model trained on in your project, or which project is the BiSeNet ckpt downloaded, thank you

Hello, Great work. Can't wait to experiment

CUDA Out Of Memory

Hello,
I need your help. I have researched and notice the solution of this error is "to reduce batch size", but I am unable to find the batch size.
can you please tell me where is batch_size defined? so that i can reduce to overcome this error.
Thanks and Cheers

In WIN10 anaconda error

e:\Anaconda3\envs\env_Sid\lib\site-packages\torch\utils\cpp_extension.py:322: UserWarning: Error checking compiler version for cl: [WinError 2] Не удается найти указанный файл
warnings.warn(f'Error checking compiler version for {compiler}: {error}')
ИНФОРМАЦИЯ: не удается найти файлы по заданным шаблонам.
Traceback (most recent call last):
File "main.py", line 13, in
from models.Embedding import Embedding
File "C:\Users\user\Barbershop\models\Embedding.py", line 3, in
from models.Net import Net
File "C:\Users\user\Barbershop\models\Net.py", line 3, in
from models.stylegan2.model import Generator
File "C:\Users\user\Barbershop\models\stylegan2\model.py", line 11, in
from models.stylegan2.op import FusedLeakyReLU, fused_leaky_relu, upfirdn2d
File "C:\Users\user\Barbershop\models\stylegan2\op_init_.py", line 1, in
from .fused_act import FusedLeakyReLU, fused_leaky_relu
File "C:\Users\user\Barbershop\models\stylegan2\op\fused_act.py", line 14, in
os.path.join(module_path, "fused_bias_act_kernel.cu"),
File "e:\Anaconda3\envs\env_Sid\lib\site-packages\torch\utils\cpp_extension.py", line 1156, in load
keep_intermediates=keep_intermediates)
File "e:\Anaconda3\envs\env_Sid\lib\site-packages\torch\utils\cpp_extension.py", line 1367, in _jit_compile
is_standalone=is_standalone)
File "e:\Anaconda3\envs\env_Sid\lib\site-packages\torch\utils\cpp_extension.py", line 1465, in _write_ninja_file_and_build_library
is_standalone=is_standalone)
File "e:\Anaconda3\envs\env_Sid\lib\site-packages\torch\utils\cpp_extension.py", line 1908, in _write_ninja_file_to_build_library
with_cuda=with_cuda)
File "e:\Anaconda3\envs\env_Sid\lib\site-packages\torch\utils\cpp_extension.py", line 2024, in _write_ninja_file
'cl']).decode(*SUBPROCESS_DECODE_ARGS).split('\r\n')
File "e:\Anaconda3\envs\env_Sid\lib\subprocess.py", line 411, in check_output
**kwargs).stdout
File "e:\Anaconda3\envs\env_Sid\lib\subprocess.py", line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['where', 'cl']' returned non-zero exit status 1.

CondaEnvException: Pip failed

When I tried to make a new environment from yml file, I got this error. I think there is typo for the version of 'clip'. Which one of 'clip' version should I choose?

Steps to Reproduce

$ conda env create -f environment/environment.yml
Collecting package metadata (repodata.json): done
Solving environment: done
Preparing transaction: done
Verifying transaction: done
Executing transaction: - By downloading and using the CUDA Toolkit conda packages, you accept the terms and conditions of the CUDA End User License Agreement (EULA): https://docs.nvidia.com/cuda/eula/index.html

done
Installing pip dependencies: \ Ran pip subprocess with arguments:
['/home/danielr/anaconda3/envs/Barbershop/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/danielr/Barbershop/environment/condaenv.b44p9c0i.requirements.txt']
Pip subprocess output:
Collecting beautifulsoup4==4.10.0
  Using cached beautifulsoup4-4.10.0-py3-none-any.whl (97 kB)
Collecting cachetools==4.2.4
  Using cached cachetools-4.2.4-py3-none-any.whl (10 kB)
Collecting charset-normalizer==2.0.7
  Using cached charset_normalizer-2.0.7-py3-none-any.whl (38 kB)
Collecting click==8.0.3
  Using cached click-8.0.3-py3-none-any.whl (97 kB)

Pip subprocess error:
ERROR: Could not find a version that satisfies the requirement clip==1.0 (from versions: 0.0.1, 0.1.0, 0.2.0)
ERROR: No matching distribution found for clip==1.0

failed

CondaEnvException: Pip failed

Expected Behavior

It should create a new environment with the specifications given in the yml file.

Environment Information

`conda info`

$ conda info

     active environment : base
    active env location : /home/danielr/anaconda3
            shell level : 1
       user config file : /home/danielr/.condarc
 populated config files : 
          conda version : 4.11.0
    conda-build version : 3.21.5
         python version : 3.9.7.final.0
       virtual packages : __cuda=11.5=0
                          __linux=5.11.0=0
                          __glibc=2.31=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /home/danielr/anaconda3  (writable)
      conda av data dir : /home/danielr/anaconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/danielr/anaconda3/pkgs
                          /home/danielr/.conda/pkgs
       envs directories : /home/danielr/anaconda3/envs
                          /home/danielr/.conda/envs
               platform : linux-64
             user-agent : conda/4.11.0 requests/2.26.0 CPython/3.9.7 Linux/5.11.0-43-generic ubuntu/20.04.3 glibc/2.31
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

`conda config --show-sources`

$ conda config --show-sources
# no output

`conda list --show-channel-urls`

$ conda list --show-channel-urls
# packages in environment at /home/danielr/anaconda3:
#
# Name                    Version                   Build  Channel
_ipyw_jlab_nb_ext_conf    0.1.0            py39h06a4308_0    defaults
_libgcc_mutex             0.1                        main    defaults
_openmp_mutex             4.5                       1_gnu    defaults
alabaster                 0.7.12             pyhd3eb1b0_0    defaults
anaconda                  2021.11                  py39_0    defaults
anaconda-client           1.9.0            py39h06a4308_0    defaults
anaconda-navigator        2.1.1                    py39_0    defaults
anaconda-project          0.10.1             pyhd3eb1b0_0    defaults
anyio                     2.2.0            py39h06a4308_1    defaults
appdirs                   1.4.4              pyhd3eb1b0_0    defaults
argh                      0.26.2           py39h06a4308_0    defaults
argon2-cffi               20.1.0           py39h27cfd23_1    defaults
arrow                     0.13.1           py39h06a4308_0    defaults
asn1crypto                1.4.0                      py_0    defaults
astroid                   2.6.6            py39h06a4308_0    defaults
astropy                   4.3.1            py39h09021b7_0    defaults
async_generator           1.10               pyhd3eb1b0_0    defaults
atomicwrites              1.4.0                      py_0    defaults
attrs                     21.2.0             pyhd3eb1b0_0    defaults
autopep8                  1.5.7              pyhd3eb1b0_0    defaults
babel                     2.9.1              pyhd3eb1b0_0    defaults
backcall                  0.2.0              pyhd3eb1b0_0    defaults
backports                 1.0                pyhd3eb1b0_2    defaults
backports.functools_lru_cache 1.6.4              pyhd3eb1b0_0    defaults
backports.shutil_get_terminal_size 1.0.0              pyhd3eb1b0_3    defaults
backports.tempfile        1.0                pyhd3eb1b0_1    defaults
backports.weakref         1.0.post1                  py_1    defaults
beautifulsoup4            4.10.0             pyh06a4308_0    defaults
binaryornot               0.4.4              pyhd3eb1b0_1    defaults
bitarray                  2.3.0            py39h7f8727e_1    defaults
bkcharts                  0.2              py39h06a4308_0    defaults
black                     19.10b0                    py_0    defaults
blas                      1.0                         mkl    defaults
bleach                    4.0.0              pyhd3eb1b0_0    defaults
blosc                     1.21.0               h8c45485_0    defaults
bokeh                     2.4.1            py39h06a4308_0    defaults
boto                      2.49.0           py39h06a4308_0    defaults
bottleneck                1.3.2            py39hdd57654_1    defaults
brotli                    1.0.9                he6710b0_2    defaults
brotlipy                  0.7.0           py39h27cfd23_1003    defaults
brunsli                   0.1                  h2531618_0    defaults
bzip2                     1.0.8                h7b6447c_0    defaults
c-ares                    1.17.1               h27cfd23_0    defaults
ca-certificates           2021.10.26           h06a4308_2    defaults
cached-property           1.5.2                      py_0    defaults
cairo                     1.16.0               hf32fb01_1    defaults
certifi                   2021.10.8        py39h06a4308_0    defaults
cffi                      1.14.6           py39h400218f_0    defaults
cfitsio                   3.470                hf0d0db6_6    defaults
chardet                   4.0.0           py39h06a4308_1003    defaults
charls                    2.2.0                h2531618_0    defaults
charset-normalizer        2.0.4              pyhd3eb1b0_0    defaults
click                     8.0.3              pyhd3eb1b0_0    defaults
cloudpickle               2.0.0              pyhd3eb1b0_0    defaults
clyent                    1.2.2            py39h06a4308_1    defaults
colorama                  0.4.4              pyhd3eb1b0_0    defaults
conda                     4.11.0           py39h06a4308_0    defaults
conda-build               3.21.5           py39h06a4308_0    defaults
conda-content-trust       0.1.1              pyhd3eb1b0_0    defaults
conda-env                 2.6.0                         1    defaults
conda-pack                0.6.0              pyhd3eb1b0_0    defaults
conda-package-handling    1.7.3            py39h27cfd23_1    defaults
conda-repo-cli            1.0.4              pyhd3eb1b0_0    defaults
conda-token               0.3.0              pyhd3eb1b0_0    defaults
conda-verify              3.4.2                      py_1    defaults
contextlib2               0.6.0.post1        pyhd3eb1b0_0    defaults
cookiecutter              1.7.2              pyhd3eb1b0_0    defaults
cryptography              3.4.8            py39hd23ed53_0    defaults
curl                      7.78.0               h1ccaba5_0    defaults
cycler                    0.10.0           py39h06a4308_0    defaults
cython                    0.29.24          py39hdbfa776_0    defaults
cytoolz                   0.11.0           py39h27cfd23_0    defaults
daal4py                   2021.3.0         py39hae6d005_0    defaults
dal                       2021.3.0           h06a4308_557    defaults
dask                      2021.10.0          pyhd3eb1b0_0    defaults
dask-core                 2021.10.0          pyhd3eb1b0_0    defaults
dataclasses               0.8                pyh6d0b6a4_7    defaults
dbus                      1.13.18              hb2f20db_0    defaults
debugpy                   1.4.1            py39h295c915_0    defaults
decorator                 5.1.0              pyhd3eb1b0_0    defaults
defusedxml                0.7.1              pyhd3eb1b0_0    defaults
diff-match-patch          20200713           pyhd3eb1b0_0    defaults
distributed               2021.10.0        py39h06a4308_0    defaults
docutils                  0.17.1           py39h06a4308_1    defaults
entrypoints               0.3              py39h06a4308_0    defaults
et_xmlfile                1.1.0            py39h06a4308_0    defaults
expat                     2.4.1                h2531618_2    defaults
fastcache                 1.1.0            py39he8ac12f_0    defaults
filelock                  3.3.1              pyhd3eb1b0_1    defaults
flake8                    3.9.2              pyhd3eb1b0_0    defaults
flask                     1.1.2              pyhd3eb1b0_0    defaults
fontconfig                2.13.1               h6c09931_0    defaults
fonttools                 4.25.0             pyhd3eb1b0_0    defaults
freetype                  2.10.4               h5ab3b9f_0    defaults
fribidi                   1.0.10               h7b6447c_0    defaults
fsspec                    2021.8.1           pyhd3eb1b0_0    defaults
future                    0.18.2           py39h06a4308_1    defaults
get_terminal_size         1.0.0                haa9412d_0    defaults
gevent                    21.8.0           py39h7f8727e_1    defaults
giflib                    5.2.1                h7b6447c_0    defaults
glib                      2.69.1               h5202010_0    defaults
glob2                     0.7                pyhd3eb1b0_0    defaults
gmp                       6.2.1                h2531618_2    defaults
gmpy2                     2.0.8            py39h8083e48_3    defaults
graphite2                 1.3.14               h23475e2_0    defaults
greenlet                  1.1.1            py39h295c915_0    defaults
gst-plugins-base          1.14.0               h8213a91_2    defaults
gstreamer                 1.14.0               h28cd5cc_2    defaults
h5py                      3.3.0            py39h930cdd6_0    defaults
harfbuzz                  2.8.1                h6f93f22_0    defaults
hdf5                      1.10.6               hb1b8bf9_0    defaults
heapdict                  1.0.1              pyhd3eb1b0_0    defaults
html5lib                  1.1                pyhd3eb1b0_0    defaults
icu                       58.2                 he6710b0_3    defaults
idna                      3.2                pyhd3eb1b0_0    defaults
imagecodecs               2021.8.26        py39h4cda21f_0    defaults
imageio                   2.9.0              pyhd3eb1b0_0    defaults
imagesize                 1.2.0              pyhd3eb1b0_0    defaults
importlib-metadata        4.8.1            py39h06a4308_0    defaults
importlib_metadata        4.8.1                hd3eb1b0_0    defaults
inflection                0.5.1            py39h06a4308_0    defaults
iniconfig                 1.1.1              pyhd3eb1b0_0    defaults
intel-openmp              2021.4.0          h06a4308_3561    defaults
intervaltree              3.1.0              pyhd3eb1b0_0    defaults
ipykernel                 6.4.1            py39h06a4308_1    defaults
ipython                   7.29.0           py39hb070fc8_0    defaults
ipython_genutils          0.2.0              pyhd3eb1b0_1    defaults
ipywidgets                7.6.5              pyhd3eb1b0_1    defaults
isort                     5.9.3              pyhd3eb1b0_0    defaults
itsdangerous              2.0.1              pyhd3eb1b0_0    defaults
jbig                      2.1                  hdba287a_0    defaults
jdcal                     1.4.1              pyhd3eb1b0_0    defaults
jedi                      0.18.0           py39h06a4308_1    defaults
jeepney                   0.7.1              pyhd3eb1b0_0    defaults
jinja2                    2.11.3             pyhd3eb1b0_0    defaults
jinja2-time               0.2.0              pyhd3eb1b0_2    defaults
joblib                    1.1.0              pyhd3eb1b0_0    defaults
jpeg                      9d                   h7f8727e_0    defaults
json5                     0.9.6              pyhd3eb1b0_0    defaults
jsonschema                3.2.0              pyhd3eb1b0_2    defaults
jupyter                   1.0.0            py39h06a4308_7    defaults
jupyter_client            6.1.12             pyhd3eb1b0_0    defaults
jupyter_console           6.4.0              pyhd3eb1b0_0    defaults
jupyter_core              4.8.1            py39h06a4308_0    defaults
jupyter_server            1.4.1            py39h06a4308_0    defaults
jupyterlab                3.2.1              pyhd3eb1b0_1    defaults
jupyterlab_pygments       0.1.2                      py_0    defaults
jupyterlab_server         2.8.2              pyhd3eb1b0_0    defaults
jupyterlab_widgets        1.0.0              pyhd3eb1b0_1    defaults
jxrlib                    1.1                  h7b6447c_2    defaults
keyring                   23.1.0           py39h06a4308_0    defaults
kiwisolver                1.3.1            py39h2531618_0    defaults
krb5                      1.19.2               hac12032_0    defaults
lazy-object-proxy         1.6.0            py39h27cfd23_0    defaults
lcms2                     2.12                 h3be6417_0    defaults
ld_impl_linux-64          2.35.1               h7274673_9    defaults
lerc                      3.0                  h295c915_0    defaults
libaec                    1.0.4                he6710b0_1    defaults
libarchive                3.4.2                h62408e4_0    defaults
libcurl                   7.78.0               h0b77cf5_0    defaults
libdeflate                1.8                  h7f8727e_5    defaults
libedit                   3.1.20210910         h7f8727e_0    defaults
libev                     4.33                 h7f8727e_1    defaults
libffi                    3.3                  he6710b0_2    defaults
libgcc-ng                 9.3.0               h5101ec6_17    defaults
libgfortran-ng            7.5.0               ha8ba4b0_17    defaults
libgfortran4              7.5.0               ha8ba4b0_17    defaults
libgomp                   9.3.0               h5101ec6_17    defaults
liblief                   0.10.1               h2531618_1    defaults
libllvm11                 11.1.0               h3826bc1_0    defaults
libnghttp2                1.41.0               hf8bcb03_2    defaults
libpng                    1.6.37               hbc83047_0    defaults
libsodium                 1.0.18               h7b6447c_0    defaults
libspatialindex           1.9.3                h2531618_0    defaults
libssh2                   1.9.0                h1ba5d50_1    defaults
libstdcxx-ng              9.3.0               hd4cf53a_17    defaults
libtiff                   4.2.0                h85742a9_0    defaults
libtool                   2.4.6             h7b6447c_1005    defaults
libuuid                   1.0.3                h7f8727e_2    defaults
libuv                     1.40.0               h7b6447c_0    defaults
libwebp                   1.2.0                h89dd481_0    defaults
libwebp-base              1.2.0                h27cfd23_0    defaults
libxcb                    1.14                 h7b6447c_0    defaults
libxml2                   2.9.12               h03d6c58_0    defaults
libxslt                   1.1.34               hc22bd24_0    defaults
libzopfli                 1.0.3                he6710b0_0    defaults
llvmlite                  0.37.0           py39h295c915_1    defaults
locket                    0.2.1            py39h06a4308_1    defaults
lxml                      4.6.3            py39h9120a33_0    defaults
lz4-c                     1.9.3                h295c915_1    defaults
lzo                       2.10                 h7b6447c_2    defaults
markupsafe                1.1.1            py39h27cfd23_0    defaults
matplotlib                3.4.3            py39h06a4308_0    defaults
matplotlib-base           3.4.3            py39hbbc1b5f_0    defaults
matplotlib-inline         0.1.2              pyhd3eb1b0_2    defaults
mccabe                    0.6.1            py39h06a4308_1    defaults
mistune                   0.8.4           py39h27cfd23_1000    defaults
mkl                       2021.4.0           h06a4308_640    defaults
mkl-service               2.4.0            py39h7f8727e_0    defaults
mkl_fft                   1.3.1            py39hd3c417c_0    defaults
mkl_random                1.2.2            py39h51133e4_0    defaults
mock                      4.0.3              pyhd3eb1b0_0    defaults
more-itertools            8.10.0             pyhd3eb1b0_0    defaults
mpc                       1.1.0                h10f8cd9_1    defaults
mpfr                      4.0.2                hb69a4c5_1    defaults
mpi                       1.0                       mpich    defaults
mpich                     3.3.2                hc856adb_0    defaults
mpmath                    1.2.1            py39h06a4308_0    defaults
msgpack-python            1.0.2            py39hff7bd54_1    defaults
multipledispatch          0.6.0            py39h06a4308_0    defaults
munkres                   1.1.4                      py_0    defaults
mypy_extensions           0.4.3            py39h06a4308_0    defaults
navigator-updater         0.2.1            py39h06a4308_0    defaults
nbclassic                 0.2.6              pyhd3eb1b0_0    defaults
nbclient                  0.5.3              pyhd3eb1b0_0    defaults
nbconvert                 6.1.0            py39h06a4308_0    defaults
nbformat                  5.1.3              pyhd3eb1b0_0    defaults
ncurses                   6.3                  heee7806_1    defaults
nest-asyncio              1.5.1              pyhd3eb1b0_0    defaults
networkx                  2.6.3              pyhd3eb1b0_0    defaults
nltk                      3.6.5              pyhd3eb1b0_0    defaults
nose                      1.3.7           pyhd3eb1b0_1006    defaults
notebook                  6.4.5            py39h06a4308_0    defaults
numba                     0.54.1           py39h51133e4_0    defaults
numexpr                   2.7.3            py39h22e1b3c_1    defaults
numpy                     1.20.3           py39hf144106_0    defaults
numpy-base                1.20.3           py39h74d4b33_0    defaults
numpydoc                  1.1.0              pyhd3eb1b0_1    defaults
olefile                   0.46               pyhd3eb1b0_0    defaults
openjpeg                  2.4.0                h3ad879b_0    defaults
openpyxl                  3.0.9              pyhd3eb1b0_0    defaults
openssl                   1.1.1l               h7f8727e_0    defaults
packaging                 21.0               pyhd3eb1b0_0    defaults
pandas                    1.3.4            py39h8c16a72_0    defaults
pandocfilters             1.4.3            py39h06a4308_1    defaults
pango                     1.45.3               hd140c19_0    defaults
parso                     0.8.2              pyhd3eb1b0_0    defaults
partd                     1.2.0              pyhd3eb1b0_0    defaults
patchelf                  0.13                 h295c915_0    defaults
path                      16.0.0           py39h06a4308_0    defaults
path.py                   12.5.0               hd3eb1b0_0    defaults
pathlib2                  2.3.6            py39h06a4308_2    defaults
pathspec                  0.7.0                      py_0    defaults
patsy                     0.5.2            py39h06a4308_0    defaults
pcre                      8.45                 h295c915_0    defaults
pep8                      1.7.1            py39h06a4308_0    defaults
pexpect                   4.8.0              pyhd3eb1b0_3    defaults
pickleshare               0.7.5           pyhd3eb1b0_1003    defaults
pillow                    8.4.0            py39h5aabda8_0    defaults
pip                       21.2.4           py39h06a4308_0    defaults
pixman                    0.40.0               h7f8727e_1    defaults
pkginfo                   1.7.1            py39h06a4308_0    defaults
pluggy                    0.13.1           py39h06a4308_0    defaults
ply                       3.11             py39h06a4308_0    defaults
poyo                      0.5.0              pyhd3eb1b0_0    defaults
prometheus_client         0.11.0             pyhd3eb1b0_0    defaults
prompt-toolkit            3.0.20             pyhd3eb1b0_0    defaults
prompt_toolkit            3.0.20               hd3eb1b0_0    defaults
psutil                    5.8.0            py39h27cfd23_1    defaults
ptyprocess                0.7.0              pyhd3eb1b0_2    defaults
py                        1.10.0             pyhd3eb1b0_0    defaults
py-lief                   0.10.1           py39h2531618_1    defaults
pycodestyle               2.7.0              pyhd3eb1b0_0    defaults
pycosat                   0.6.3            py39h27cfd23_0    defaults
pycparser                 2.20                       py_2    defaults
pycurl                    7.44.1           py39h8f2d780_1    defaults
pydocstyle                6.1.1              pyhd3eb1b0_0    defaults
pyerfa                    2.0.0            py39h27cfd23_0    defaults
pyflakes                  2.3.1              pyhd3eb1b0_0    defaults
pygments                  2.10.0             pyhd3eb1b0_0    defaults
pyjwt                     2.1.0            py39h06a4308_0    defaults
pylint                    2.9.6            py39h06a4308_1    defaults
pyls-spyder               0.4.0              pyhd3eb1b0_0    defaults
pyodbc                    4.0.31           py39h295c915_0    defaults
pyopenssl                 21.0.0             pyhd3eb1b0_1    defaults
pyparsing                 3.0.4              pyhd3eb1b0_0    defaults
pyqt                      5.9.2            py39h2531618_6    defaults
pyrsistent                0.18.0           py39heee7806_0    defaults
pysocks                   1.7.1            py39h06a4308_0    defaults
pytables                  3.6.1            py39h77479fe_1    defaults
pytest                    6.2.4            py39h06a4308_2    defaults
python                    3.9.7                h12debd9_1    defaults
python-dateutil           2.8.2              pyhd3eb1b0_0    defaults
python-libarchive-c       2.9                pyhd3eb1b0_1    defaults
python-lsp-black          1.0.0              pyhd3eb1b0_0    defaults
python-lsp-jsonrpc        1.0.0              pyhd3eb1b0_0    defaults
python-lsp-server         1.2.4              pyhd3eb1b0_0    defaults
python-slugify            5.0.2              pyhd3eb1b0_0    defaults
pytz                      2021.3             pyhd3eb1b0_0    defaults
pywavelets                1.1.1            py39h6323ea4_4    defaults
pyxdg                     0.27               pyhd3eb1b0_0    defaults
pyyaml                    6.0              py39h7f8727e_1    defaults
pyzmq                     22.2.1           py39h295c915_1    defaults
qdarkstyle                3.0.2              pyhd3eb1b0_0    defaults
qstylizer                 0.1.10             pyhd3eb1b0_0    defaults
qt                        5.9.7                h5867ecd_1    defaults
qtawesome                 1.0.2              pyhd3eb1b0_0    defaults
qtconsole                 5.1.1              pyhd3eb1b0_0    defaults
qtpy                      1.10.0             pyhd3eb1b0_0    defaults
readline                  8.1                  h27cfd23_0    defaults
regex                     2021.8.3         py39h7f8727e_0    defaults
requests                  2.26.0             pyhd3eb1b0_0    defaults
ripgrep                   12.1.1                        0    defaults
rope                      0.19.0             pyhd3eb1b0_0    defaults
rtree                     0.9.7            py39h06a4308_1    defaults
ruamel_yaml               0.15.100         py39h27cfd23_0    defaults
scikit-image              0.18.3           py39h51133e4_0    defaults
scikit-learn              0.24.2           py39ha9443f7_0    defaults
scikit-learn-intelex      2021.3.0         py39h06a4308_0    defaults
scipy                     1.7.1            py39h292c36d_2    defaults
seaborn                   0.11.2             pyhd3eb1b0_0    defaults
secretstorage             3.3.1            py39h06a4308_0    defaults
send2trash                1.8.0              pyhd3eb1b0_1    defaults
setuptools                58.0.4           py39h06a4308_0    defaults
simplegeneric             0.8.1            py39h06a4308_2    defaults
singledispatch            3.7.0           pyhd3eb1b0_1001    defaults
sip                       4.19.13          py39h2531618_0    defaults
six                       1.16.0             pyhd3eb1b0_0    defaults
snappy                    1.1.8                he6710b0_0    defaults
sniffio                   1.2.0            py39h06a4308_1    defaults
snowballstemmer           2.1.0              pyhd3eb1b0_0    defaults
sortedcollections         2.1.0              pyhd3eb1b0_0    defaults
sortedcontainers          2.4.0              pyhd3eb1b0_0    defaults
soupsieve                 2.2.1              pyhd3eb1b0_0    defaults
sphinx                    4.2.0              pyhd3eb1b0_1    defaults
sphinxcontrib             1.0              py39h06a4308_1    defaults
sphinxcontrib-applehelp   1.0.2              pyhd3eb1b0_0    defaults
sphinxcontrib-devhelp     1.0.2              pyhd3eb1b0_0    defaults
sphinxcontrib-htmlhelp    2.0.0              pyhd3eb1b0_0    defaults
sphinxcontrib-jsmath      1.0.1              pyhd3eb1b0_0    defaults
sphinxcontrib-qthelp      1.0.3              pyhd3eb1b0_0    defaults
sphinxcontrib-serializinghtml 1.1.5              pyhd3eb1b0_0    defaults
sphinxcontrib-websupport  1.2.4                      py_0    defaults
spyder                    5.1.5            py39h06a4308_1    defaults
spyder-kernels            2.1.3            py39h06a4308_0    defaults
sqlalchemy                1.4.22           py39h7f8727e_0    defaults
sqlite                    3.36.0               hc218d9a_0    defaults
statsmodels               0.12.2           py39h27cfd23_0    defaults
sympy                     1.9              py39h06a4308_0    defaults
tbb                       2021.4.0             hd09550d_0    defaults
tbb4py                    2021.4.0         py39hd09550d_0    defaults
tblib                     1.7.0              pyhd3eb1b0_0    defaults
terminado                 0.9.4            py39h06a4308_0    defaults
testpath                  0.5.0              pyhd3eb1b0_0    defaults
text-unidecode            1.3                pyhd3eb1b0_0    defaults
textdistance              4.2.1              pyhd3eb1b0_0    defaults
threadpoolctl             2.2.0              pyh0d69192_0    defaults
three-merge               0.1.1              pyhd3eb1b0_0    defaults
tifffile                  2021.7.2           pyhd3eb1b0_2    defaults
tinycss                   0.4             pyhd3eb1b0_1002    defaults
tk                        8.6.11               h1ccaba5_0    defaults
toml                      0.10.2             pyhd3eb1b0_0    defaults
toolz                     0.11.1             pyhd3eb1b0_0    defaults
tornado                   6.1              py39h27cfd23_0    defaults
tqdm                      4.62.3             pyhd3eb1b0_1    defaults
traitlets                 5.1.0              pyhd3eb1b0_0    defaults
typed-ast                 1.4.3            py39h7f8727e_1    defaults
typing_extensions         3.10.0.2           pyh06a4308_0    defaults
tzdata                    2021e                hda174b7_0    defaults
ujson                     4.0.2            py39h2531618_0    defaults
unicodecsv                0.14.1           py39h06a4308_0    defaults
unidecode                 1.2.0              pyhd3eb1b0_0    defaults
unixodbc                  2.3.9                h7b6447c_0    defaults
urllib3                   1.26.7             pyhd3eb1b0_0    defaults
watchdog                  2.1.3            py39h06a4308_0    defaults
wcwidth                   0.2.5              pyhd3eb1b0_0    defaults
webencodings              0.5.1            py39h06a4308_1    defaults
werkzeug                  2.0.2              pyhd3eb1b0_0    defaults
wheel                     0.37.0             pyhd3eb1b0_1    defaults
whichcraft                0.6.1              pyhd3eb1b0_0    defaults
widgetsnbextension        3.5.1            py39h06a4308_0    defaults
wrapt                     1.12.1           py39he8ac12f_1    defaults
wurlitzer                 2.1.1            py39h06a4308_0    defaults
xlrd                      2.0.1              pyhd3eb1b0_0    defaults
xlsxwriter                3.0.1              pyhd3eb1b0_0    defaults
xlwt                      1.3.0            py39h06a4308_0    defaults
xmltodict                 0.12.0             pyhd3eb1b0_0    defaults
xz                        5.2.5                h7b6447c_0    defaults
yaml                      0.2.5                h7b6447c_0    defaults
yapf                      0.31.0             pyhd3eb1b0_0    defaults
zeromq                    4.3.4                h2531618_0    defaults
zfp                       0.5.5                h2531618_6    defaults
zict                      2.0.0              pyhd3eb1b0_0    defaults
zipp                      3.6.0              pyhd3eb1b0_0    defaults
zlib                      1.2.11               h7b6447c_3    defaults
zope                      1.0              py39h06a4308_1    defaults
zope.event                4.5.0            py39h06a4308_0    defaults
zope.interface            5.4.0            py39h7f8727e_0    defaults
zstd                      1.4.9                haebb681_0    defaults

wait for experiment!!!

@ZPdesu

Cant create an environment from the environment.yml

Collecting package metadata (repodata.json): done
Solving environment: failed

ResolvePackageNotFound:

• pyqt==5.9.2=py37h05f1152_2
• setuptools==52.0.0=py37h06a4308_0
• pyyaml==5.4.1=py37h27cfd23_1
• pyrsistent==0.17.3=py37h7b6447c_0
• pcre==8.45=h295c915_0
• numpy-base==1.20.3=py37h74d4b33_0
• xz==5.2.5=h7b6447c_0
• readline==8.1=h27cfd23_0
• notebook==6.4.3=py37h06a4308_0
• ipython==7.26.0=py37hb070fc8_0
• numpy==1.20.3=py37hf144106_0
• jupyter_core==4.7.1=py37h06a4308_0
• libsodium==1.0.18=h7b6447c_0
• libuuid==1.0.3=h1bed415_2
• libidn2==2.3.2=h7f8727e_0
• libgfortran-ng==7.3.0=hdf63c60_0
• scipy==1.6.2=py37had2a1c9_1
• cytoolz==0.11.0=py37h7b6447c_0
• zlib==1.2.11=h7b6447c_3
• ipykernel==6.2.0=py37h06a4308_1
• zeromq==4.3.4=h2531618_0
• ffmpeg==4.3=hf484d3e_0
• freetype==2.10.4=h5ab3b9f_0
• jpeg==9b=h024ee3a_2
• dbus==1.13.18=hb2f20db_0
• kiwisolver==1.3.1=py37h2531618_0
• pyzmq==22.2.1=py37h295c915_1
• mistune==0.8.4=py37h14c3975_1001
• certifi==2021.10.8=py37h06a4308_0
• fontconfig==2.13.1=h6c09931_0
• lz4-c==1.9.3=h295c915_1
• pandocfilters==1.4.3=py37h06a4308_1
• openjpeg==2.3.0=h05c96fa_1
• bzip2==1.0.8=h7b6447c_0
• debugpy==1.4.1=py37h295c915_0
• lame==3.100=h7b6447c_0
• gmp==6.2.1=h2531618_2
• libxcb==1.14=h7b6447c_0
• zstd==1.4.9=haebb681_0
• openssl==1.1.1l=h7f8727e_0
• libpng==1.6.37=hbc83047_0
• tifffile==2020.10.1=py37hdd07704_2
• pillow==8.3.1=py37h2c7a002_0
• libiconv==1.15=h63c8f33_5
• mkl==2021.3.0=h06a4308_520
• qt==5.9.7=h5867ecd_1
• libstdcxx-ng==9.1.0=hdf63c60_0
• sqlite==3.36.0=hc218d9a_0
• gst-plugins-base==1.14.0=h8213a91_2
• mkl-service==2.4.0=py37h7f8727e_0
• libunistring==0.9.10=h27cfd23_0
• python==3.7.11=h12debd9_0
• jedi==0.18.0=py37h06a4308_1
• tornado==6.1=py37h27cfd23_0
• expat==2.4.1=h2531618_2
• libwebp-base==1.2.0=h27cfd23_0
• openh264==2.1.0=hd408876_0
• glib==2.69.1=h5202010_0
• scikit-image==0.18.1=py37ha9443f7_0
• cudatoolkit==11.0.221=h6bb024c_0
• mkl_random==1.2.2=py37h51133e4_0
• matplotlib-base==3.3.4=py37h62a2d02_0
• sip==4.19.8=py37hf484d3e_0
• lcms2==2.12=h3be6417_0
• pip==21.2.2=py37h06a4308_0
• nbconvert==6.1.0=py37h06a4308_0
• mkl_fft==1.3.0=py37h42c9631_2
• libffi==3.3=he6710b0_2
• argon2-cffi==20.1.0=py37h27cfd23_1
• libtiff==4.2.0=h85742a9_0
• nettle==3.7.3=hbbd107a_1
• libxml2==2.9.10=hb55368b_3
• ninja==1.10.2=hff7bd54_1
• gnutls==3.6.15=he1e5248_0
• libuv==1.40.0=h7b6447c_0
• cffi==1.14.6=py37h400218f_0
• ld_impl_linux-64==2.35.1=h7274673_9
• importlib-metadata==3.10.0=py37h06a4308_0
• icu==58.2=he6710b0_3
• yaml==0.2.5=h7b6447c_0
• ncurses==6.2=he6710b0_1
• ca-certificates==2021.10.26=h06a4308_2
• gstreamer==1.14.0=h28cd5cc_2
• libtasn1==4.16.0=h27cfd23_0
• markupsafe==2.0.1=py37h27cfd23_0
• tk==8.6.10=hbc83047_0
• libgcc-ng==9.1.0=hdf63c60_0
• intel-openmp==2021.3.0=h06a4308_3350
• pywavelets==1.1.1=py37h7b6447c_2
• pytorch==1.7.1=py3.7_cuda11.0.221_cudnn8.0.5_0
• terminado==0.9.4=py37h06a4308_0
• locket==0.2.1=py37h06a4308_1

How to change image size?

Hey @ZPdesu ,

I love your project and currently working on some test cases. I want to improve the speed of embeddings by using smaller pictures. Is there a easy to achieve this?

I used
python align_face.py -output_size 512
to generate the pictures with size 512

after running
python main.py --im_path1 90.png --im_path2 20.png --im_path3 20.png --sign realistic --smooth 5 --size 512

I got this error:

RuntimeError: Error(s) in loading state_dict for Generator:
        Unexpected key(s) in state_dict: "convs.14.conv.weight", "convs.14.conv.blur.kernel", "convs.14.conv.modulation.weight", "convs.14.conv.modulation.bias", "convs.14.noise.weight", "convs.14.activate.bias", "convs.15.conv.weight", "convs.15.conv.modulation.weight", "convs.15.conv.modulation.bias", "convs.15.noise.weight", "convs.15.activate.bias", "to_rgbs.7.bias", "to_rgbs.7.upsample.kernel", "to_rgbs.7.conv.weight", "to_rgbs.7.conv.modulation.weight", "to_rgbs.7.conv.modulation.bias", "noises.noise_15", "noises.noise_16".

OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.

I tried to run the main.py by following the steps in ReadMe and got this error. What should I do to fix this?

Steps to Reproduce

$ python main.py --im_path1 90.png --im_path2 15.png --im_path3 117.png --sign realistic --smooth 5
Traceback (most recent call last):
  File "main.py", line 13, in <module>
    from models.Embedding import Embedding
  File "/home/danielr/Barbershop/models/Embedding.py", line 3, in <module>
    from models.Net import Net
  File "/home/danielr/Barbershop/models/Net.py", line 3, in <module>
    from models.stylegan2.model import Generator
  File "/home/danielr/Barbershop/models/stylegan2/model.py", line 11, in <module>
    from models.stylegan2.op import FusedLeakyReLU, fused_leaky_relu, upfirdn2d
  File "/home/danielr/Barbershop/models/stylegan2/op/__init__.py", line 1, in <module>
    from .fused_act import FusedLeakyReLU, fused_leaky_relu
  File "/home/danielr/Barbershop/models/stylegan2/op/fused_act.py", line 14, in <module>
    os.path.join(module_path, "fused_bias_act_kernel.cu"),
  File "/home/danielr/anaconda3/envs/Barbershop/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 997, in load
    keep_intermediates=keep_intermediates)
  File "/home/danielr/anaconda3/envs/Barbershop/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1202, in _jit_compile
    with_cuda=with_cuda)
  File "/home/danielr/anaconda3/envs/Barbershop/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1279, in _write_ninja_file_and_build_library
    verbose)
  File "/home/danielr/anaconda3/envs/Barbershop/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1369, in _prepare_ldflags
    extra_ldflags.append('-L{}'.format(_join_cuda_home('lib64')))
  File "/home/danielr/anaconda3/envs/Barbershop/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1827, in _join_cuda_home
    raise EnvironmentError('CUDA_HOME environment variable is not set. '
OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.

Expected Behavior

It should produce the image results.

Environment Information

`conda info`

$ conda info

     active environment : Barbershop
    active env location : /home/danielr/anaconda3/envs/Barbershop
            shell level : 2
       user config file : /home/danielr/.condarc
 populated config files : 
          conda version : 4.11.0
    conda-build version : 3.21.5
         python version : 3.9.7.final.0
       virtual packages : __cuda=11.5=0
                          __linux=5.11.0=0
                          __glibc=2.31=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /home/danielr/anaconda3  (writable)
      conda av data dir : /home/danielr/anaconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/danielr/anaconda3/pkgs
                          /home/danielr/.conda/pkgs
       envs directories : /home/danielr/anaconda3/envs
                          /home/danielr/.conda/envs
               platform : linux-64
             user-agent : conda/4.11.0 requests/2.26.0 CPython/3.9.7 Linux/5.11.0-43-generic ubuntu/20.04.3 glibc/2.31
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

`conda config --show-sources`

$ conda config --show-sources 
# no output

`conda list --show-channel-urls`

$ conda list --show-channel-urls 
# packages in environment at /home/danielr/anaconda3/envs/Barbershop:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main    defaults
argcomplete               1.12.3             pyhd3eb1b0_0    defaults
argon2-cffi               20.1.0           py37h27cfd23_1    defaults
async_generator           1.10               pyhd3eb1b0_0    defaults
attrs                     21.2.0             pyhd3eb1b0_0    defaults
backcall                  0.2.0              pyhd3eb1b0_0    defaults
beautifulsoup4            4.10.0                   pypi_0    pypi
blas                      1.0                         mkl    defaults
bleach                    4.0.0              pyhd3eb1b0_0    defaults
bzip2                     1.0.8                h7b6447c_0    defaults
ca-certificates           2021.10.26           h06a4308_2    defaults
cachetools                4.2.4                    pypi_0    pypi
certifi                   2021.10.8        py37h06a4308_0    defaults
cffi                      1.14.6           py37h400218f_0    defaults
charset-normalizer        2.0.7                    pypi_0    pypi
click                     8.0.3                    pypi_0    pypi
cloudpickle               1.6.0                      py_0    defaults
cudatoolkit               11.0.221             h6bb024c_0    defaults
cycler                    0.10.0                   py37_0    defaults
cytoolz                   0.11.0           py37h7b6447c_0    defaults
dask-core                 2021.8.0           pyhd3eb1b0_0    defaults
dbus                      1.13.18              hb2f20db_0    defaults
debugpy                   1.4.1            py37h295c915_0    defaults
decorator                 5.0.9              pyhd3eb1b0_0    defaults
defusedxml                0.7.1              pyhd3eb1b0_0    defaults
deprecated                1.2.13                   pypi_0    pypi
dlib                      19.22.1                  pypi_0    pypi
entrypoints               0.3                      py37_0    defaults
et-xmlfile                1.1.0                    pypi_0    pypi
expat                     2.4.1                h2531618_2    defaults
ffmpeg                    4.3                  hf484d3e_0    pytorch
filelock                  3.4.0                    pypi_0    pypi
fontconfig                2.13.1               h6c09931_0    defaults
freetype                  2.10.4               h5ab3b9f_0    defaults
fsspec                    2021.7.0           pyhd3eb1b0_0    defaults
ftfy                      6.0.3                    pypi_0    pypi
gdown                     4.2.0                    pypi_0    pypi
glib                      2.69.1               h5202010_0    defaults
gmp                       6.2.1                h2531618_2    defaults
gnutls                    3.6.15               he1e5248_0    defaults
google-api-core           1.31.4                   pypi_0    pypi
google-api-python-client  2.7.0                    pypi_0    pypi
google-auth               1.35.0                   pypi_0    pypi
google-auth-httplib2      0.1.0                    pypi_0    pypi
googleapis-common-protos  1.54.0                   pypi_0    pypi
gst-plugins-base          1.14.0               h8213a91_2    defaults
gstreamer                 1.14.0               h28cd5cc_2    defaults
httplib2                  0.19.1                   pypi_0    pypi
huggingface-hub           0.1.2                    pypi_0    pypi
icu                       58.2                 he6710b0_3    defaults
idna                      3.3                      pypi_0    pypi
imageio                   2.9.0              pyhd3eb1b0_0    defaults
importlib-metadata        3.10.0           py37h06a4308_0    defaults
importlib_metadata        3.10.0               hd3eb1b0_0    defaults
intel-openmp              2021.3.0          h06a4308_3350    defaults
ipykernel                 6.2.0            py37h06a4308_1    defaults
ipython                   7.26.0           py37hb070fc8_0    defaults
ipython_genutils          0.2.0              pyhd3eb1b0_1    defaults
ipywidgets                7.6.3              pyhd3eb1b0_1    defaults
jedi                      0.18.0           py37h06a4308_1    defaults
jinja2                    3.0.1              pyhd3eb1b0_0    defaults
joblib                    1.1.0                    pypi_0    pypi
jpeg                      9b                   h024ee3a_2    defaults
jsonschema                3.2.0                      py_2    defaults
jupyter                   1.0.0                    py37_7    defaults
jupyter_client            6.1.12             pyhd3eb1b0_0    defaults
jupyter_console           6.4.0              pyhd3eb1b0_0    defaults
jupyter_core              4.7.1            py37h06a4308_0    defaults
jupyterlab_pygments       0.1.2                      py_0    defaults
jupyterlab_widgets        1.0.0              pyhd3eb1b0_1    defaults
kiwisolver                1.3.1            py37h2531618_0    defaults
lame                      3.100                h7b6447c_0    defaults
lcms2                     2.12                 h3be6417_0    defaults
ld_impl_linux-64          2.35.1               h7274673_9    defaults
libffi                    3.3                  he6710b0_2    defaults
libgcc-ng                 9.1.0                hdf63c60_0    defaults
libgfortran-ng            7.3.0                hdf63c60_0    defaults
libiconv                  1.15                 h63c8f33_5    defaults
libidn2                   2.3.2                h7f8727e_0    defaults
libpng                    1.6.37               hbc83047_0    defaults
libsodium                 1.0.18               h7b6447c_0    defaults
libstdcxx-ng              9.1.0                hdf63c60_0    defaults
libtasn1                  4.16.0               h27cfd23_0    defaults
libtiff                   4.2.0                h85742a9_0    defaults
libunistring              0.9.10               h27cfd23_0    defaults
libuuid                   1.0.3                h1bed415_2    defaults
libuv                     1.40.0               h7b6447c_0    defaults
libwebp-base              1.2.0                h27cfd23_0    defaults
libxcb                    1.14                 h7b6447c_0    defaults
libxml2                   2.9.10               hb55368b_3    defaults
locket                    0.2.1            py37h06a4308_1    defaults
lz4-c                     1.9.3                h295c915_1    defaults
markupsafe                2.0.1            py37h27cfd23_0    defaults
matplotlib-base           3.3.4            py37h62a2d02_0    defaults
matplotlib-inline         0.1.2              pyhd3eb1b0_2    defaults
mistune                   0.8.4           py37h14c3975_1001    defaults
mkl                       2021.3.0           h06a4308_520    defaults
mkl-service               2.4.0            py37h7f8727e_0    defaults
mkl_fft                   1.3.0            py37h42c9631_2    defaults
mkl_random                1.2.2            py37h51133e4_0    defaults
nbclient                  0.5.3              pyhd3eb1b0_0    defaults
nbconvert                 6.1.0            py37h06a4308_0    defaults
nbformat                  5.1.3              pyhd3eb1b0_0    defaults
ncurses                   6.2                  he6710b0_1    defaults
nest-asyncio              1.5.1              pyhd3eb1b0_0    defaults
nettle                    3.7.3                hbbd107a_1    defaults
networkx                  2.6.2              pyhd3eb1b0_0    defaults
ninja                     1.10.2               hff7bd54_1    defaults
notebook                  6.4.3            py37h06a4308_0    defaults
numpy                     1.20.3           py37hf144106_0    defaults
numpy-base                1.20.3           py37h74d4b33_0    defaults
oauth2client              4.1.3                    pypi_0    pypi
olefile                   0.46                       py_0    defaults
opencv-python             4.5.4.58                 pypi_0    pypi
openh264                  2.1.0                hd408876_0    defaults
openjpeg                  2.3.0                h05c96fa_1    defaults
openpyxl                  3.0.7                    pypi_0    pypi
openssl                   1.1.1l               h7f8727e_0    defaults
packaging                 21.0               pyhd3eb1b0_0    defaults
pandocfilters             1.4.3            py37h06a4308_1    defaults
parso                     0.8.2              pyhd3eb1b0_0    defaults
partd                     1.2.0              pyhd3eb1b0_0    defaults
pcre                      8.45                 h295c915_0    defaults
pexpect                   4.8.0              pyhd3eb1b0_3    defaults
pickleshare               0.7.5           pyhd3eb1b0_1003    defaults
pillow                    8.3.1            py37h2c7a002_0    defaults
pip                       21.2.2           py37h06a4308_0    defaults
prometheus_client         0.11.0             pyhd3eb1b0_0    defaults
prompt-toolkit            3.0.17             pyh06a4308_0    defaults
prompt_toolkit            3.0.17               hd3eb1b0_0    defaults
protobuf                  3.19.1                   pypi_0    pypi
ptyprocess                0.7.0              pyhd3eb1b0_2    defaults
pyasn1                    0.4.8                    pypi_0    pypi
pyasn1-modules            0.2.8                    pypi_0    pypi
pycparser                 2.20                       py_2    defaults
pygments                  2.10.0             pyhd3eb1b0_0    defaults
pyparsing                 2.4.7              pyhd3eb1b0_0    defaults
pyqt                      5.9.2            py37h05f1152_2    defaults
pyqt5                     5.13.0                   pypi_0    pypi
pyqt5-qt5                 5.15.2                   pypi_0    pypi
pyqt5-sip                 12.9.0                   pypi_0    pypi
pyrsistent                0.17.3           py37h7b6447c_0    defaults
pysocks                   1.7.1                    pypi_0    pypi
python                    3.7.11               h12debd9_0    defaults
python-dateutil           2.8.2              pyhd3eb1b0_0    defaults
python-magic              0.4.24                   pypi_0    pypi
pytorch                   1.7.1           py3.7_cuda11.0.221_cudnn8.0.5_0    pytorch
pytorch-fid               0.2.1                    pypi_0    pypi
pytorch-msssim            0.2.1                    pypi_0    pypi
pytz                      2021.3                   pypi_0    pypi
pywavelets                1.1.1            py37h7b6447c_2    defaults
pyyaml                    5.4.1            py37h27cfd23_1    defaults
pyzmq                     22.2.1           py37h295c915_1    defaults
qdarkgraystyle            1.0.2                    pypi_0    pypi
qdarkstyle                3.0.2                    pypi_0    pypi
qt                        5.9.7                h5867ecd_1    defaults
qtconsole                 5.1.0              pyhd3eb1b0_0    defaults
qtpy                      1.10.0             pyhd3eb1b0_0    defaults
readline                  8.1                  h27cfd23_0    defaults
regex                     2021.11.2                pypi_0    pypi
requests                  2.26.0                   pypi_0    pypi
rsa                       4.8                      pypi_0    pypi
sacremoses                0.0.46                   pypi_0    pypi
scikit-image              0.18.1           py37ha9443f7_0    defaults
scikit-learn              1.0.1                    pypi_0    pypi
scipy                     1.6.2            py37had2a1c9_1    defaults
send2trash                1.5.0              pyhd3eb1b0_1    defaults
setuptools                52.0.0           py37h06a4308_0    defaults
sip                       4.19.8           py37hf484d3e_0    defaults
six                       1.16.0             pyhd3eb1b0_0    defaults
soupsieve                 2.3.1                    pypi_0    pypi
sqlite                    3.36.0               hc218d9a_0    defaults
terminado                 0.9.4            py37h06a4308_0    defaults
testpath                  0.5.0              pyhd3eb1b0_0    defaults
threadpoolctl             3.0.0                    pypi_0    pypi
tifffile                  2020.10.1        py37hdd07704_2    defaults
tk                        8.6.10               hbc83047_0    defaults
tokenizers                0.10.3                   pypi_0    pypi
toolz                     0.11.1             pyhd3eb1b0_0    defaults
torchaudio                0.7.2                      py37    pytorch
torchdiffeq               0.2.2                    pypi_0    pypi
torchvision               0.8.2                py37_cu110    pytorch
tornado                   6.1              py37h27cfd23_0    defaults
tqdm                      4.62.3                   pypi_0    pypi
traitlets                 5.0.5              pyhd3eb1b0_0    defaults
transformers              4.12.3                   pypi_0    pypi
typing_extensions         3.10.0.0           pyh06a4308_0    defaults
uritemplate               3.0.1                    pypi_0    pypi
urllib3                   1.26.7                   pypi_0    pypi
wcwidth                   0.2.5                      py_0    defaults
webencodings              0.5.1                    py37_1    defaults
wheel                     0.37.0             pyhd3eb1b0_0    defaults
widgetsnbextension        3.5.1                    py37_0    defaults
wrapt                     1.13.3                   pypi_0    pypi
xz                        5.2.5                h7b6447c_0    defaults
yaml                      0.2.5                h7b6447c_0    defaults
zeromq                    4.3.4                h2531618_0    defaults
zipp                      3.5.0              pyhd3eb1b0_0    defaults
zlib                      1.2.11               h7b6447c_3    defaults
zstd                      1.4.9                haebb681_0    defaults

add web demo/model to Huggingface

Hi, would you be interested in adding Barbershop to Hugging Face? The Hub offers free hosting, and it would make your work more accessible and visible to the rest of the ML community. Models/datasets/spaces(web demos) can be added to a user account or organization similar to github.

Example from other organizations:
Keras: https://huggingface.co/keras-io
Microsoft: https://huggingface.co/microsoft
Facebook: https://huggingface.co/facebook

Example spaces with repos:
github: https://github.com/salesforce/BLIP
Spaces: https://huggingface.co/spaces/salesforce/BLIP

github: https://github.com/facebookresearch/omnivore
Spaces: https://huggingface.co/spaces/akhaliq/omnivore

and here are guides for adding spaces/models/datasets to your org

How to add a Space: https://huggingface.co/blog/gradio-spaces
how to add models: https://huggingface.co/docs/hub/adding-a-model
uploading a dataset: https://huggingface.co/docs/datasets/upload_dataset.html

Please let us know if you would be interested and if you have any questions, we can also help with the technical implementation.

Support for multi-gpus

great work guys, I was just curious if there is going to be support for multiple gpu's in the future.

Thanks

pip install version problem with package clip

Hello, first and foremost thank you very much for sharing the code, the work is incredibile and we are extremely lucky to be able to freely use and try it.

I've succesfully ran the code on a VM on google colab (linux) by creating a conda env from the provided yaml file however it looks like that the version of package clip==1.0 might be wrong as it can't pip install it.

I've simply removed the version and everything works perfectly. Just wanted to let you guys now!

Have a good day

How to get output results?

I have cloned the repository and installed all the dependencies.
I have two images, img_1, and img_2. I want to swap img_1 hairs with img_2 hairs, how can I do it? I run the first main.py command, but couldn't find the results. If somebody can point out what I am missing?
@ZPdesu

Can this project change and modify the hairstyles of yellow men?(I tried to process photos of Chinese male stars, but the results were not good)

Can't run main.py with sample images

Hi, I set up the environment on google Colab and trying to test-run the code.

I first ran the below code and got image 90.png ready.

python align_face.py

And, when I try to run the code I'm getting the following error:

python main.py --im_path1 90.png --im_path2 15.png --im_path3 117.png --sign realistic --smooth 5

Align Step 2:   0%|          | 0/100 [00:00<?, ?it/s]/usr/local/envs/Barbershop/lib/python3.7/site-packages/torch/nn/functional.py:3103: UserWarning: The default behavior for interpolate/upsample with float scale_factor changed in 1.6.0 to align with other frameworks/libraries, and now uses scale_factor directly, instead of relying on the computed output size. If you wish to restore the old behavior, please set recompute_scale_factor=True. See the documentation of nn.Upsample for details. 
  warnings.warn("The default behavior for interpolate/upsample with float scale_factor changed "
/usr/local/envs/Barbershop/lib/python3.7/site-packages/torch/nn/functional.py:3063: UserWarning: Default upsampling behavior when mode=bicubic is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  "See the documentation of nn.Upsample for details.".format(mode))

I tried changing align_corners & recompute_scale_factor and that did not fix the problem.

How could I possibly solve the problem?

What does this line of code mean : gen_im_0_1 = (gen_im + 1) / 2

I'm very confused why we need to add 1 and divide 2 here

How to change face expression

The expression has changed like in the thesi

How to convert trained ffhq.pt model to coreML model for iOS

I am having difficulty converting my trained ffhq.pt model to coreML model for my iOS application. I'll appreciate if anyone can help. Unable to understand which architecture was being used and how to load and then convert into coreML model. Any help would be appreciated. Thanks

Can't detect face in aligne_face.py

I'm trying to use Barbershop with a custom image. The image shot is HQ, jpg format and it's very similar to the one provided by default in the /unprocessed folder (90.jpg).
The problem is that the detector is not able to find the face in it, i checked in:

def get_landmark(filepath,predictor):
    """get landmark with dlib
    :return: np.array shape=(68, 2)"""
    
    detector = dlib.get_frontal_face_detector()
    img = dlib.load_rgb_image(filepath)
    print("img",img)
    dets = detector(img, 1)
    print("dets",dets)  

img [[[176 177 182]
  [176 177 182]
  [177 178 183]
  ...
  [ 67  69  68]
  [100 102 101]
  [125 127 126]]

 [[177 178 183]
  [177 178 183]
  [177 178 183]
  ...
  [ 62  64  63]
  [104 106 105]
  [126 128 127]]

 [[178 179 184]
  [178 179 184]
  [178 179 184]
  ...
  [ 77  79  78]
  [114 116 115]
  [116 118 117]]

 ...

 [[168 168 166]
  [168 168 166]
  [169 169 167]
  ...
  [170 121  88]
  [170 121  88]
  [170 121  88]]

 [[168 168 166]
  [168 168 166]
  [169 169 167]
  ...
  [172 123  90]
  [172 123  90]
  [172 123  90]]

 [[168 168 166]
  [168 168 166]
  [169 169 167]
  ...
  [173 124  91]
  [173 124  91]
  [171 122  89]]]
dets rectangles[]  

As you can see rectangles is empty, i tried to set dets = detector(img, 1) and dets = detector(img, 0) with no avail, how can i try to make it work with a custom picture?
Thanks in advance

Suggestions for hosting this on AWS EC2 instance

Do I need to make sure I get an instance type with GPUs in order to use CUDA?

Any other considerations for how to set this up to run on EC2?

I Found No file named 'train.py' 'test.py'

Hello,I am very glad to see such a great work! I'm just a DL new man. But I have noticed there is no 'train.py' and 'test.py' in the mode/readme.md. I am looking forward to learn the more details. Thanks for your attention!

i got a type error "download() got an unexpected keyword argument 'fuzzy' "

i was trying your model and entered the code below

!python main.py --im_path1 90.png --im_path2 15.png --im_path3 117.png --sign realistic --smooth 5

Then I got this 'fuzzy' argument error.

Downloading StyleGAN2 checkpoint: pretrained_models/ffhq.pt
Traceback (most recent call last):
File "main.py", line 117, in
main(args)
File "main.py", line 18, in main
ii2s = Embedding(args)
File "/content/drive/My Drive/StyleGAN_dir/Barbershop/models/Embedding.py", line 23, in init
self.net = Net(self.opts)
File "/content/drive/My Drive/StyleGAN_dir/Barbershop/models/Net.py", line 15, in init
self.load_weights()
File "/content/drive/My Drive/StyleGAN_dir/Barbershop/models/Net.py", line 22, in load_weights
download_weight(self.opts.ckpt)
File "/content/drive/My Drive/StyleGAN_dir/Barbershop/utils/model_utils.py", line 17, in download_weight
output=weight_path, fuzzy=True)
TypeError: download() got an unexpected keyword argument 'fuzzy'

I couldn't find this 'fuzzy' argument you used in gdown.download method.
Can you tell me what this is?

please, there is no pretrained model I can find.can you give me a link

Question About Initial Segmentation

Thanks for your work! It's awesome!
Could you release the weight of the face segmentation network? It would be great if it could be released.

Hello, boss, can't this project be used on Windows system

Errors are reported when creating a virtual environment in Anaconda

`ResolvePackageNotFound:

• gnutls=3.6.15
• libxcb=1.14
• gmp=6.2.1
• libgfortran-ng=7.3.0
• libtasn1=4.16.0
• libstdcxx-ng=9.1.0
• libidn2=2.3.2
• libunistring=0.9.10
• lame=3.100
• gstreamer=1.14.0
• libuuid=1.0.3
• nettle=3.7.3
• ncurses=6.2
• gst-plugins-base=1.14.0
• libgcc-ng=9.1.0
• openh264=2.1.0
• dbus=1.13.18
• ld_impl_linux-64=2.35.1
• readline=8.1`

In WIN10 anaconda error

e:\Anaconda3\envs\env_Sid\lib\site-packages\torch\utils\cpp_extension.py:322: UserWarning: Error checking compiler version for cl: [WinError 2] Не удается найти указанный файл
warnings.warn(f'Error checking compiler version for {compiler}: {error}')
ИНФОРМАЦИЯ: не удается найти файлы по заданным шаблонам.
Traceback (most recent call last):
File "main.py", line 13, in
from models.Embedding import Embedding
File "C:\Users\user\Barbershop\models\Embedding.py", line 3, in
from models.Net import Net
File "C:\Users\user\Barbershop\models\Net.py", line 3, in
from models.stylegan2.model import Generator
File "C:\Users\user\Barbershop\models\stylegan2\model.py", line 11, in
from models.stylegan2.op import FusedLeakyReLU, fused_leaky_relu, upfirdn2d
File "C:\Users\user\Barbershop\models\stylegan2\op_init_.py", line 1, in
from .fused_act import FusedLeakyReLU, fused_leaky_relu
File "C:\Users\user\Barbershop\models\stylegan2\op\fused_act.py", line 14, in
os.path.join(module_path, "fused_bias_act_kernel.cu"),
File "e:\Anaconda3\envs\env_Sid\lib\site-packages\torch\utils\cpp_extension.py", line 1156, in load
keep_intermediates=keep_intermediates)
File "e:\Anaconda3\envs\env_Sid\lib\site-packages\torch\utils\cpp_extension.py", line 1367, in _jit_compile
is_standalone=is_standalone)
File "e:\Anaconda3\envs\env_Sid\lib\site-packages\torch\utils\cpp_extension.py", line 1465, in _write_ninja_file_and_build_library
is_standalone=is_standalone)
File "e:\Anaconda3\envs\env_Sid\lib\site-packages\torch\utils\cpp_extension.py", line 1908, in _write_ninja_file_to_build_library
with_cuda=with_cuda)
File "e:\Anaconda3\envs\env_Sid\lib\site-packages\torch\utils\cpp_extension.py", line 2024, in _write_ninja_file
'cl']).decode(*SUBPROCESS_DECODE_ARGS).split('\r\n')
File "e:\Anaconda3\envs\env_Sid\lib\subprocess.py", line 411, in check_output
**kwargs).stdout
File "e:\Anaconda3\envs\env_Sid\lib\subprocess.py", line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['where', 'cl']' returned non-zero exit status 1.