Coder Social home page Coder Social logo

barisgecer / ostec Goto Github PK

View Code? Open in Web Editor NEW
189.0 189.0 28.0 46.1 MB

TF implementation of our CVPR 2021 paper: OSTeC: One-Shot Texture Completion

Home Page: https://openaccess.thecvf.com/content/CVPR2021/html/Gecer_OSTeC_One-Shot_Texture_Completion_CVPR_2021_paper.html

License: Other

Python 100.00%
3d-face-reconstruction 3dmm face-frontalization texture-completion texture-synthesis

ostec's People

Contributors

barisgecer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ostec's Issues

Training Step

Hello, I am very interested in your work. What if I wanted to train this network from scratch? Can you give me detailed training procedures?

Deep3Dfacerecon

Can you help me please to solve this problem? I try to run test.py for Deep3Dfacerecon.

----------------- Options ---------------
add_image: True
bfm_folder: BFM
bfm_model: BFM_model_front.mat
camera_d: 10.0
center: 112.0
checkpoints_dir: ./checkpoints
dataset_mode: None
ddp_port: 12355
display_per_batch: True
epoch: latest
eval_batch_nums: inf
focal: 1015.0
gpu_ids: 0
img_folder: examples
init_path: checkpoints/init_model/resnet50-0676ba61.pth
isTrain: False [default: None]
model: facerecon
name: face_recon
net_recon: resnet50
phase: test
suffix:
use_ddp: False [default: True]
use_last_fc: False
verbose: False
vis_batch_nums: 1
world_size: 1
z_far: 15.0
z_near: 5.0
----------------- End -------------------
model [FaceReconModel] was created
loading the model from ./checkpoints/face_recon/epoch_latest.pth
0 examples/000002.jpg
Traceback (most recent call last):
File "test.py", line 72, in
main(0, opt,opt.img_folder)
File "test.py", line 62, in main
model.test() # run inference
File "/content/drive/MyDrive/Deep3DFaceRecon_pytorch-baris/models/base_model.py", line 162, in test
self.forward()
File "/content/drive/MyDrive/Deep3DFaceRecon_pytorch-baris/models/facerecon_model.py", line 137, in forward
self.facemodel.compute_for_render(output_coeff)
File "/content/drive/MyDrive/Deep3DFaceRecon_pytorch-baris/models/bfm.py", line 303, in compute_for_render
face_color = self.compute_color(face_texture, face_norm_roted, coef_dict['gamma'])
File "/content/drive/MyDrive/Deep3DFaceRecon_pytorch-baris/models/bfm.py", line 176, in compute_color
face_color = torch.cat([r, g, b], dim=-1) * face_texture
RuntimeError: The size of tensor a (35709) must match the size of tensor b (53215) at non-singleton dimension 1

tensorflow_core/include/Eigen/Core:28:12: fatal error: cuda/std/complex: No such file or directory

is anyone meet this problem?
I install tensorflow 1.15.5 on Ubuntu, CUDA 11.1, cudnn 8
In file included from .conda/envs/ostecP/lib/python3.6/site-packages/tensorflow_core/include/third_party/eigen3/Eigen/Core:1,
from .conda/envs/ostecP/lib/python3.6/site-packages/tensorflow_core/include/tensorflow/core/lib/strings/strcat.h:29,
from .conda/envs/ostecP/lib/python3.6/site-packages/tensorflow_core/include/tensorflow/core/lib/core/errors.h:24,
from ~/OSTeC/external/stylegan2/dnnlib/tflib/ops/tensorflow/core/framework/op.h:26,
from external/stylegan2/dnnlib/tflib/ops/fused_bias_act.cu:9:
.conda/envs/ostecP/lib/python3.6/site-packages/tensorflow_core/include/Eigen/Core:28:12: fatal error: cuda/std/complex: No such file or directory
28 | #include <cuda/std/complex>
| ^~~~~~~~~~~~~~~~~~
compilation terminated.

Problem of generating obj file with texture mapping

First of all, thank you very much for sharing such excellent work. I want to put the last generated texture into meshlab for analysis, so I generated an obj file, which contains 3D face vertices and corresponding faces, but there was a problem when adding 2D texture vertices and faces. The complete 2D faces index is not found. Can you tell me how to get the complete 2D texture vertices and faces? Thank you again! Looking forward to your reply!

IndexError: index 0 is out of bounds for axis 0 with size 0`

Hello, some input images have such an error, do you know why? Hope to get an answer, thank you very much!

Traceback (most recent call last):
File "run_ostec.py", line 98, in
main(args)
File "run_ostec.py", line 73, in main
final_uv, results_dict = operator.run(img, fitting, face_mask)
File "/data/code/OSTeC/core/operator.py", line 351, in run
face = self.get_tmesh(im, reconstruction_dict, face_mask)
File "/data/code/OSTeC/core/operator.py", line 207, in get_tmesh
im_masked = fill_UV(im_PIL2menpo(im_masked))
File "/data/code/OSTeC/utils/utils.py", line 132, in fill_UV
result = interp(np.ravel(xx[mask]), np.ravel(yy[mask]))
File "/data/anaconda3/envs/ostec/lib/python3.6/site-packages/scipy/interpolate/ndgriddata.py", line 79, in call
return self.values[i]
IndexError: index 0 is out of bounds for axis 0 with size 0

i get ResolvePackageNotFound:error i'm trying on Windows

conda env create -f environment.yml -n ostec
Collecting package metadata (repodata.json): done
Solving environment: failed

ResolvePackageNotFound:

  • dbus==1.13.6=h48d8840_2
  • hdf5==1.10.6=nompi_h6a2412b_1114
  • menpo3d==0.8.3=py36h355b2fd_0
  • gst-plugins-base==1.18.4=hf529b03_2
  • libgomp==9.3.0=h5101ec6_17
  • ffmpeg==4.3.2=hca11adc_0
  • glew==2.1.0=h9c3ff4c_2
  • bzip2==1.0.8=h7f98852_4
  • libogg==1.3.4=h7f98852_1
  • xorg-xextproto==7.3.0=h7f98852_1002
  • pyqt-impl==5.12.3=py36h7ec31b9_7
  • freetype==2.10.4=h0708190_1
  • loguru==0.5.3=py36h5fab9bb_2
  • gstreamer==1.18.4=h76c114f_2
  • libssh2==1.9.0=ha56f1ee_6
  • pyqtchart==5.12=py36h7ec31b9_7
  • c-ares==1.17.1=h7f98852_1
  • qt==5.12.9=hda022c4_4
  • cytoolz==0.11.0=py36h8f6f2f9_3
  • scikit-image==0.17.2=py36h284efc9_4
  • harfbuzz==2.8.2=h83ec7ef_0
  • libstdcxx-ng==9.3.0=hd4cf53a_17
  • alsa-lib==1.2.3=h516909a_0
  • _openmp_mutex==4.5=1_gnu
  • mysql-common==8.0.25=ha770c72_2
  • curl==7.78.0=hea6ffbf_0
  • libevent==2.1.10=hcdb4288_3
  • libdeflate==1.7=h7f98852_5
  • openjpeg==2.4.0=hb52868f_1
  • xorg-libxt==1.2.1=h7f98852_2
  • libvorbis==1.3.7=h9c3ff4c_0
  • pthread-stubs==0.4=h36c2ea0_1001
  • pyqtwebengine==5.12.1=py36h7ec31b9_7
  • certifi==2020.12.5=py36h06a4308_0
  • libcblas==3.9.0=11_linux64_openblas
  • py-opencv==4.5.2=py36hcb3619a_1
  • xorg-libxext==1.3.4=h7f98852_1
  • gl2ps==1.4.2=h0708190_0
  • libgcc-ng==9.3.0=h5101ec6_17
  • moderngl==5.6.4=py36h284efc9_0
  • gettext==0.19.8.1=h0b5b191_1005
  • libgfortran-ng==11.2.0=h69a702a_11
  • libxkbcommon==1.0.3=he3ba5ed_0
  • readline==8.1=h27cfd23_0
  • fontconfig==2.13.1=hba837de_1005
  • xorg-libxau==1.0.9=h7f98852_0
  • opencv==4.5.2=py36h5fab9bb_1
  • libxcb==1.13=h7f98852_1003
  • pillow==8.3.1=py36h676a545_0
  • libcurl==7.78.0=h2574ce0_0
  • krb5==1.19.2=hcc1bbae_0
  • libev==4.33=h516909a_1
  • menpo==0.11.0=py36h355b2fd_0
  • setuptools==58.0.4=py36h06a4308_0
  • graphite2==1.3.13=h58526e2_1001
  • libopenblas==0.3.17=pthreads_h8fe5266_1
  • xz==5.2.5=h7b6447c_0
  • lz4-c==1.9.3=h9c3ff4c_1
  • libopus==1.3.1=h7f98852_1
  • expat==2.4.1=h9c3ff4c_0
  • tornado==6.1=py36h8f6f2f9_1
  • xorg-xproto==7.0.31=h7f98852_1007
  • traits==6.2.0=py36h8f6f2f9_0
  • x264==1!161.3030=h7f98852_1
  • liblapacke==3.9.0=11_linux64_openblas
  • zstd==1.5.0=ha95c52a_0
  • openh264==2.1.1=h780b84a_0
  • cyvlfeat==0.7.0=py36h0280710_0
  • libnetcdf==4.8.0=nompi_hcd642e3_103
  • xorg-libxrender==0.9.10=h7f98852_1003
  • libpng==1.6.37=h21135ba_2
  • lcms2==2.12=hddcbb42_0
  • jpeg==9d=h36c2ea0_0
  • libtheora==1.1.1=h7f98852_1005
  • xorg-kbproto==1.0.7=h7f98852_1002
  • jasper==1.900.1=h07fcdf6_1006
  • libglib==2.68.3=h3e27bee_0
  • icu==68.1=h58526e2_0
  • libblas==3.9.0=11_linux64_openblas
  • libpq==13.3=hd57d9b9_0
  • libtiff==4.3.0=hf544144_1
  • tbb==2020.2=h4bd325d_4
  • xorg-libice==1.0.10=h7f98852_0
  • gmp==6.2.1=h58526e2_0
  • pugixml==1.11.4=h9c3ff4c_0
  • cairo==1.16.0=h6cf1ce9_1008
  • ca-certificates==2021.9.30=h06a4308_1
  • libclang==11.1.0=default_ha53f305_1
  • nettle==3.6=he412f7d_0
  • openssl==1.1.1l=h7f8727e_0
  • double-conversion==3.1.5=h9c3ff4c_2
  • proj==7.2.0=h277dcde_2
  • libuuid==2.32.1=h7f98852_1000
  • yaml==0.2.5=h516909a_0
  • libgfortran5==11.2.0=h5c6108e_11
  • xorg-renderproto==0.11.1=h7f98852_1002
  • glib-tools==2.68.3=h9c3ff4c_0
  • immutables==0.15=py36h8f6f2f9_0
  • zlib==1.2.11=h7b6447c_3
  • glcontext==2.3.4=py36hc4f0c31_0
  • tk==8.6.11=h1ccaba5_0
  • python==3.6.13=h12debd9_1
  • nss==3.67=hb5efdd6_0
  • pip==21.0.1=py36h06a4308_0
  • pywavelets==1.1.1=py36h92226af_3
  • pixman==0.40.0=h36c2ea0_0
  • libglu==9.0.0=he1b5a44_1001
  • libopencv==4.5.2=py36hb84549a_1
  • mysql-libs==8.0.25=hfa10184_2
  • tbb-devel==2020.2=h4bd325d_4
  • kiwisolver==1.3.1=py36h605e78d_1
  • utfcpp==3.2.1=ha770c72_0
  • libffi==3.3=he6710b0_2
  • scipy==1.5.3=py36h9e8f40b_0
  • libxml2==2.9.12=h72842e0_0
  • vtk==9.0.1=no_osmesa_py36hfa3a401_109
  • libiconv==1.16=h516909a_0
  • lame==3.100=h7f98852_1001
  • pyqt5-sip==4.19.18=py36hc4f0c31_7
  • lerc==2.2.1=h9c3ff4c_0
  • jsoncpp==1.9.4=h4bd325d_3
  • pyqt==5.12.3=py36h5fab9bb_7
  • pathlib==1.0.1=py36h5fab9bb_4
  • ncurses==6.2=he6710b0_1
  • liblapack==3.9.0=11_linux64_openblas
  • nspr==4.30=h9c3ff4c_0
  • h5py==2.10.0=py36hd6299e0_1
  • importlib-metadata==4.8.1=py36h5fab9bb_0
  • xorg-libx11==1.7.2=h7f98852_0
  • gnutls==3.6.13=h85f3911_1
  • xorg-libsm==1.2.3=hd9c2040_1000
  • eigen==3.3.9=h4bd325d_1
  • libllvm11==11.1.0=hf817b99_2
  • libprotobuf==3.16.0=h780b84a_0
  • pcre==8.45=h9c3ff4c_0
  • libzip==1.8.0=h4de3113_0
  • xorg-libxdmcp==1.1.3=h7f98852_0
  • libwebp-base==1.2.0=h7f98852_2
  • libedit==3.1.20191231=he28a2e2_2
  • libnghttp2==1.43.0=h812cca2_0
  • vlfeat==0.9.20=h14c3975_1002
  • ld_impl_linux-64==2.35.1=h7274673_9
  • glib==2.68.3=h9c3ff4c_0
  • imagecodecs-lite==2019.12.3=py36h92226af_3
  • jbig==2.1=h7f98852_2003
  • sqlite==3.36.0=hc218d9a_0
  • hdf4==4.2.15=h10796ff_3

Will the repository work with CUDA 11?

I have NVIDIA-SMI 510.47.03 Driver Version: 510.47.03 CUDA Version: 11.6 installed on the server. I followed the instructions to install the repository by installing cuda 10.0, cudnn 7.4 and tensorflow 1.14 inside ostec environment. But after all the models get loaded, I get this error while processing images.

Started: ./test_imgs/0000_0099_00.png Traceback (most recent call last): File "run_ostec.py", line 96, in <module> main(args) File "run_ostec.py", line 66, in main fitting = deep3dmodel.recontruct(im_menpo2PIL(img), lms) File "/home/suvidha/av-project/OSTeC/external/deep3dfacerecon/ostec_api.py", line 68, in recontruct coeffs = self.model.net_recon(self.model.input_img) File "/home/suvidha/anaconda3/envs/ostec/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "external/deep3dfacerecon/models/networks.py", line 98, in forward x = self.backbone(x) File "/home/suvidha/anaconda3/envs/ostec/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "external/deep3dfacerecon/models/networks.py", line 375, in forward return self._forward_impl(x) File "external/deep3dfacerecon/models/networks.py", line 358, in _forward_impl x = self.conv1(x) File "/home/suvidha/anaconda3/envs/ostec/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "/home/suvidha/anaconda3/envs/ostec/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 423, in forward return self._conv_forward(input, self.weight) File "/home/suvidha/anaconda3/envs/ostec/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 420, in _conv_forward self.padding, self.dilation, self.groups) RuntimeError: CUDA error: no kernel image is available for execution on the device

Please help me with the solution. I am stuck and unable to make this repository work since 2 days now.
I also can't find cuda path (installed using conda) which I can export in bashrc.

errors

active conda and all that
when I install everything and proceed to run the script

python run_ostec.py --source_dir [source_dir] --save_dir [save_dir] [-f] -i [iterations (default 200)] -m [soft | hard | auto]

first error

Traceback (most recent call last):
  File "run_ostec.py", line 12, in <module>
    import menpo.io as mio
ModuleNotFoundError: No module named 'menpo'

then! pip install menpo

then another mistake


/usr/local/lib/python3.7/site-packages/menpo/image/base.py:25: UserWarning: Falling back to scipy interpolation for affine warps
  warn("Falling back to scipy interpolation for affine warps")
Traceback (most recent call last):
  File "run_ostec.py", line 17, in <module>
    from core.operator import Operator
  File "/content/OSTeC/core/operator.py", line 7, in <module>
    from core.projection_handler import Projection_Handler
  File "/content/OSTeC/core/projection_handler.py", line 13, in <module>
    import external.stylegan2.dnnlib.tflib as tflib
  File "/content/OSTeC/external/stylegan2/dnnlib/tflib/__init__.py", line 7, in <module>
    from . import autosummary
  File "/content/OSTeC/external/stylegan2/dnnlib/tflib/autosummary.py", line 26, in <module>
    import tensorflow as tf
ModuleNotFoundError: No module named 'tensorflow'

which version of tensorflow should I install

It seems that Conda did not install some things so I installed it myself but now it throws this error

/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
Using TensorFlow backend.
/usr/local/lib/python3.7/site-packages/kornia/augmentation/augmentation.py:1875: DeprecationWarning: GaussianBlur is no longer maintained and will be removed from the future versions. Please use RandomGaussianBlur instead.
  category=DeprecationWarning,
Traceback (most recent call last):
  File "/content/OSTeC/run_ostec.py", line 74, in <module>
    main(args)
  File "/content/OSTeC/run_ostec.py", line 27, in main
    operator = Operator(args)
  File "/content/OSTeC/core/operator.py", line 94, in __init__
    self.tcoords_full = mio.import_pickle('models/topology/tcoords_full.pkl')
  File "/usr/local/lib/python3.7/site-packages/menpo/io/input/base.py", line 484, in import_pickle
    return _import(filepath, pickle_types, importer_kwargs=kwargs)
  File "/usr/local/lib/python3.7/site-packages/menpo/io/input/base.py", line 912, in _import
    raise ValueError("{} is not a file".format(path))
ValueError: /content/OSTeC/external/deep3dfacerecon/models/topology/tcoords_full.pkl is not a file

Error in conda environment

Hello, can anyone please share their working conda environment + cuda and cudnn + python versions. I've installed everything according to the instructions and also installed the environment with the provided yml but still cant run the code.

run parameter -f unrecognized

Thanks for your great job. I

  1. When I run the code it tips -f unrecongnized arguments.
  2. Another problem is that I can not see any differences from the output datas between the different arguments with hard or soft.
  3. Could you give some suggestions about how to handle with multiple images

Crashing while running the program: same configuration that this repository was tested with

Hi,
I followed your installation and environment requirements.
I pass all the model loading and compilations successfully, but then in line 95 in file arcface_handler.py:
return self.sess.run(self.emb_norm, {self.img_ph: [img.pixels_with_channels_at_back()], self.lms_ph:[lms]})
it crashes with error:
Check failed: cusolverDnCreate(&cusolver_dn_handle) == CUSOLVER_STATUS_SUCCESS Failed to create cuSolverDN instance
any idea why this happens? (tried to google it, but did not find anything that was not in the code already as config.gpu_options.allow_growth = True
Any idea how I can fix this?
Thanks

no output

Hi Baris, thanks for sharing your work, I tried to run the code ( in google colab). the FaceRecon Model was created, also the test_opt.txt. but there is no result (images).

Using TensorFlow backend.
2022-02-14 16:52:33.113699: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2022-02-14 16:52:33.119043: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2299995000 Hz
2022-02-14 16:52:33.119314: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5644886c7d40 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2022-02-14 16:52:33.119366: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2022-02-14 16:52:33.121564: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2022-02-14 16:52:33.252205: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:33.253213: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x56448856bd40 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2022-02-14 16:52:33.253252: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Tesla K80, Compute Capability 3.7
2022-02-14 16:52:33.253528: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:33.254265: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235
pciBusID: 0000:00:04.0
2022-02-14 16:52:33.254699: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2022-02-14 16:52:33.256152: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2022-02-14 16:52:33.257373: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2022-02-14 16:52:33.257785: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2022-02-14 16:52:33.259474: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2022-02-14 16:52:33.260793: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2022-02-14 16:52:33.264730: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2022-02-14 16:52:33.264851: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:33.265688: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:33.266405: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2022-02-14 16:52:33.266498: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2022-02-14 16:52:33.268258: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2022-02-14 16:52:33.268321: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0
2022-02-14 16:52:33.268339: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N
2022-02-14 16:52:33.268511: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:33.269298: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:33.270104: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10813 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)
2022-02-14 16:52:35.228822: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:35.229581: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235
pciBusID: 0000:00:04.0
2022-02-14 16:52:35.229682: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2022-02-14 16:52:35.229742: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2022-02-14 16:52:35.229792: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2022-02-14 16:52:35.229851: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2022-02-14 16:52:35.229896: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2022-02-14 16:52:35.229942: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2022-02-14 16:52:35.229987: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2022-02-14 16:52:35.230102: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:35.230899: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:35.231592: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2022-02-14 16:52:35.231650: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2022-02-14 16:52:35.231673: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0
2022-02-14 16:52:35.231689: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N
2022-02-14 16:52:35.231805: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:35.232608: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:35.233400: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10813 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)
Setting up TensorFlow plugin "fused_bias_act.cu": Preprocessing... 2022-02-14 16:52:36.641169: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:36.641936: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235
pciBusID: 0000:00:04.0
2022-02-14 16:52:36.642055: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2022-02-14 16:52:36.642117: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2022-02-14 16:52:36.642165: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2022-02-14 16:52:36.642255: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2022-02-14 16:52:36.642314: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2022-02-14 16:52:36.642360: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2022-02-14 16:52:36.642407: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2022-02-14 16:52:36.642506: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:36.643388: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:36.644114: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2022-02-14 16:52:36.644167: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2022-02-14 16:52:36.644188: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0
2022-02-14 16:52:36.644202: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N
2022-02-14 16:52:36.644335: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:36.645056: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:36.645756: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/device:GPU:0 with 10813 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)
Loading... Done.
Setting up TensorFlow plugin "upfirdn_2d.cu": Preprocessing... 2022-02-14 16:52:37.640597: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:37.641460: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235
pciBusID: 0000:00:04.0
2022-02-14 16:52:37.641571: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2022-02-14 16:52:37.641625: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2022-02-14 16:52:37.641674: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2022-02-14 16:52:37.641726: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2022-02-14 16:52:37.641774: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2022-02-14 16:52:37.641821: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2022-02-14 16:52:37.641871: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2022-02-14 16:52:37.641986: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:37.642752: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:37.643515: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2022-02-14 16:52:37.643576: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2022-02-14 16:52:37.643600: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0
2022-02-14 16:52:37.643616: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N
2022-02-14 16:52:37.643758: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:37.644520: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-14 16:52:37.645297: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/device:GPU:0 with 10813 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)
Loading... Done.
2022-02-14 16:52:51.391704: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2022-02-14 16:52:51.524127: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
Loading ResNet Model:
2022-02-14 16:54:07.694218: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2022-02-14 16:54:07.694277: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]
MTCNN loaded
model [FaceReconModel] was created
loading the model from /content/drive/MyDrive/OSTeC1/external/deep3dfacerecon/checkpoints/face_recon/epoch_latest.pth
Scanning paths...
Scanning paths...

Problem of "_pickle.UnpicklingError: pickle data was truncated"

First of all, thank you very much for sharing such excellent work! When I complete the installation, and run "python run_ostec.py --source_dir [source_dir] --save_dir [save_dir] [-f] -i [iterations (default 200)] -m [soft|hard|auto]", the error occurrs:

`Using TensorFlow backend.
/////miniconda3/envs/ostec/lib/python3.6/site-packages/kornia/augmentation/augmentation.py:1875: DeprecationWarning: GaussianBlur is no longer maintained and will be removed from the future versions. Please use RandomGaussianBlur instead.
category=DeprecationWarning,
Traceback (most recent call last):
File "/////OSTeC/external/arcface50.py", line 19, in load_weights
weights_dict = np.load(weight_file).item()
File "/////miniconda3/envs/ostec/lib/python3.6/site-packages/numpy/lib/npyio.py", line 440, in load
pickle_kwargs=pickle_kwargs)
File "/////miniconda3/envs/ostec/lib/python3.6/site-packages/numpy/lib/format.py", line 693, in read_array
array = pickle.load(fp, **pickle_kwargs)
_pickle.UnpicklingError: pickle data was truncated

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "run_ostec.py", line 76, in
main(args)
File "run_ostec.py", line 28, in main
operator = Operator(args)
File "/////OSTeC/core/operator.py", line 112, in init
self.arcface_handler = Arcface_Handler()
File "/////OSTeC/core/arcface_handler.py", line 85, in init
self.emb_norm, _, _, vars = identity_features(aligned_img, 'input_id_features')
File "/////OSTeC/core/arcface_handler.py", line 74, in identity_features
input, embedding, content = arcface50.KitModel('models/fr_models/arcface50.npy', input * 255)
File "/////OSTeC/external/arcface50.py", line 28, in KitModel
__weights_dict = load_weights(weight_file)
File "/////OSTeC/external/arcface50.py", line 21, in load_weights
weights_dict = np.load(weight_file, encoding='bytes').item()
File "/////miniconda3/envs/ostec/lib/python3.6/site-packages/numpy/lib/npyio.py", line 440, in load
pickle_kwargs=pickle_kwargs)
File "/////miniconda3/envs/ostec/lib/python3.6/site-packages/numpy/lib/format.py", line 693, in read_array
array = pickle.load(fp, **pickle_kwargs)
_pickle.UnpicklingError: pickle data was truncated`

Looking for your reply. Thank you very much!

args parameters

I keep getting error because of args parameters is not defiend
can you share args parameters to start with?

nvcc fatal : Host compiler targets unsupported OS.

when I run the command: python run_ostec.py --source_dir MyTestData/Input --save_dir MyTestData/Output -i 100 -m soft
occur a error : nvcc fatal : Host compiler targets unsupported OS.
I don't know how to solve it

How to align I_i , M_i , and S′_i to the alignment template of StyleGANv2?

Hi, Baris!
I love your job, wonderful job!! However, there is a question about how to align the S′_i and M_i to aligned rendering picture I_i.
It's obvious that I_i can be aligned by converting it to PIL image and feeding into the ffhq dataset face alignment pipeline. But there is no clue about how to align the projected face model S'_i to the aligned rendering picture to make S'_i to be landmark correspondent to I_i.
I would be very appreciate if you can answer my question. Thank you again for such amazing job!!
Best regards,
Rodney.

ModuleNotFoundError: No module named 'models.arcface_torch'

python run_ostec.py --source_dir /content/9.jpg --save_dir /content/out [-f] -i 200 -m hard

/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
Using TensorFlow backend.
Traceback (most recent call last):
  File "run_ostec.py", line 21, in <module>
    from external.deep3dfacerecon.ostec_api import Deep3dModel
  File "/content/OSTeC/external/deep3dfacerecon/ostec_api.py", line 5, in <module>
    from models import create_model
  File "external/deep3dfacerecon/models/__init__.py", line 22, in <module>
    from models.base_model import BaseModel
  File "external/deep3dfacerecon/models/base_model.py", line 9, in <module>
    from . import networks
  File "external/deep3dfacerecon/models/networks.py", line 18, in <module>
    from .arcface_torch.backbones import get_model
ModuleNotFoundError: No module named 'models.arcface_torch'

how to generate obj file?

I want to observe the reconstructed performance in meshlab or blender, please give me a tip how to generate the obj file, thank a lot.

help: exporting mesh?

Hello, I tried to capture the trimesh generated by Operator#run:

face = self.get_tmesh(im, reconstruction_dict)
results_dict['face'] = face

and then export it using menpo3d io:

import menpo3d.io as m3dio
...
m3dio.export_textured_mesh(results_dict["face"].tmesh, save_path.replace(ext, ".obj"), extension="obj", overwrite=True)

but the generated mesh is corrupt:

image

any idea why? is there a proper way to do so? tried using original code from Deep3DFaceRecon_pytorch and the mesh is ok but it fails to generate texture due tensor size missmatch.

Render 4-channel image?

Hi, the shape of im_masked is (3, 1120, 1120), and then concats with the 2d face segmentation mask mask_landmarks. What does the renderer do to the four-channel texture map? It will render RGB and mask channel seperately? What the new mask img_uv_src.pixels[3]<0.5 mean?

OSTeC/core/operator.py

Lines 288 to 293 in ebf348e

im_masked = fill_UV(im_PIL2menpo(im_masked))
im_masked.pixels = np.concatenate([im_masked.pixels, np.expand_dims(mask_landmarks,0)],0)
img_uv_src = self.render_uv_image(im_masked, dense_lms[self.mask])
mask_landmarks = img_uv_src.pixels[3]<0.5
img_uv_src.pixels = img_uv_src.pixels[0:3]
img_uv_src = fill_UV(img_uv_src)

Thanks.

How to render correctly?

input:
ffhq

render results after reconstruction:
ffhq_render_angle_0
ffhq_render_angle_45

code:

# fitting
fitting = deep3dmodel.recontruct(im_menpo2PIL(img), lms, test=False)

# Prepare Textured Trimesh with visible part of the face
face = operator.get_tmesh(im, fitting)

# Render to specific angle
from utils.image_rasterization import rasterize_image

im, _ = rasterize_image(face.tmesh, operator.img_shape, pose_angle_deg=[0, 0, 0], cam_dist=4.5)
cv2.imwrite(f'./split_test/ffhq_render_angle_0.jpg', np.array(im_menpo2PIL(im))[:, :, ::-1])

im, _ = rasterize_image(face.tmesh, operator.img_shape, pose_angle_deg=[0, 45, 0], cam_dist=4.5)
cv2.imwrite(f'./split_test/ffhq_render_angle_45.jpg', np.array(im_menpo2PIL(im))[:, :, ::-1])

problem for the 'environment.yml'

I found that the configuration file should be exported from Linux. Who can provide the configuration file suitable for Windows? Thank you very much.

Unable to reproduce paper results

As per the subject line; tried to frontalize the photo of Christopher Lee (top left input in Fig.1), the result looks nothing like yours for some reason...

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.