parthaeth / gif Goto Github PK

I followed your instruction "First things first" and got error:

git clone --recurse-submodules [email protected]:ParthaEth/GIF.git
Cloning into 'GIF'...
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Is there something wrong?

line in generate_gif.py from my_utils.ringnet_overlay.util import tensor_vis_landmarks, There is no such folder in the repository. Could you please upload it?

I've tested generate_sample_random.py. The image quality is very nice but, the generated faces doesn't look like the same person.
Does anyone know what should we change the parts of code to generate images of similar appearances but different poses like fig.1 in the paper?

Dear authors, thank you so much for sharing your work. I wanted to reproduce the results by training the model from scratch. I have installed the packages in the requirements.txt in a python 3.8.18 environment but I am getting error like:

Exception has occurred: RuntimeError       (note: full exception trace is shown but execution is paused at: _run_module_as_main)
Not compiled with GPU support
  File "/data/mdwkhan/3d_gen_codes/pytorch3d/pytorch3d/renderer/mesh/rasterize_meshes.py", line 189, in forward
    pix_to_face, zbuf, barycentric_coords, dists = _C.rasterize_meshes(
  File "/data/mdwkhan/3d_gen_codes/pytorch3d/pytorch3d/renderer/mesh/rasterize_meshes.py", line 136, in rasterize_meshes
    return _RasterizeFaceVerts.apply(
  File "/data/mdwkhan/3d_gen_codes/GIF/my_utils/photometric_optimization/renderer.py", line 59, in forward
    pix_to_face, zbuf, bary_coords, dists = rasterize_meshes(
  File "/data/mdwkhan/anaconda3/envs/gif2/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/data/mdwkhan/3d_gen_codes/GIF/my_utils/photometric_optimization/renderer.py", line 152, in forward
    rendering = self.rasterizer(transformed_vertices, self.faces.expand(batch_size, -1, -1), attributes)
  File "/data/mdwkhan/anaconda3/envs/gif2/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/data/mdwkhan/3d_gen_codes/GIF/my_utils/photometric_optimization/gif_helper.py", line 37, in render_tex_and_normal
    rendering_results = self.render(verts, trans_verts, albedos, lights=lightcode)
  File "/data/mdwkhan/3d_gen_codes/GIF/my_utils/visualize_flame_overlay.py", line 24, in get_rendered_mesh
    self.rendering_helper.render_tex_and_normal(shapecode=shape, expcode=expression,
  File "/data/mdwkhan/3d_gen_codes/GIF/loss_functions/losses.py", line 210, in get_image_and_textures
    self.flame_visualizer.get_rendered_mesh(flame_params=(shape, exp, pose, light_code, texture_code),
  File "/data/mdwkhan/3d_gen_codes/GIF/loss_functions/losses.py", line 163, in tex_sp_intrp_loss
    textures, tx_masks, _ = self.get_image_and_textures(alpha, flame_batch, generator, max_ids, normal_maps_as_cond,
  File "/data/mdwkhan/3d_gen_codes/GIF/train.py", line 229, in train
    interp_loss = interp_tex_loss.tex_sp_intrp_loss(
  File "/data/mdwkhan/3d_gen_codes/GIF/train.py", line 402, in <module>
    train(args, dataset, generator, discriminator_flm, fid_computer, flame_param_est, used_sampless,
  File "/data/mdwkhan/anaconda3/envs/gif2/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/data/mdwkhan/anaconda3/envs/gif2/lib/python3.8/runpy.py", line 194, in _run_module_as_main (Current frame)
    return _run_code(code, main_globals, None,
RuntimeError: Not compiled with GPU support

Hi, thanks for your great work!
I find that DECA can only preocess images with size 224*224 ? How do you get the flame parameters of FFHQ which are 1024 *1024?
Hope to get your guidance！

Hi! I've followed the "First thing first" instructions. Now I'd like to generate some images, without any training, so I continued with the "To run random face generation" instructions, but I get this error

python role_of_different_parameters.py
Traceback (most recent call last):
File "role_of_different_parameters.py", line 7, in
from model.stg2_generator import StyledGenerator
File "../model/stg2_generator.py", line 15, in
from my_utils.photometric_optimization.models import FLAME
ModuleNotFoundError: No module named 'my_utils.photometric_optimization.models'

hello
when I run generate_random_samples.py I found this warning.Please tell me where I can find this file.Thanks
/usr/local/lib/python3.6/dist-packages/pytorch3d/io/obj_io.py:457: UserWarning: Mtl file does not exist: /content/GIF/GIF_resources/input_files/flame_resource/template.mtl
warnings.warn(f"Mtl file does not exist: {f}")

Thinks for your great job, it's brings new ideas to this research topic.
However, I find that GIF_resources file is missing, and the link to the file is not public available.
Could you please release necessary files recently? Can't wait to run your code.

Thanks.

Thank you for sharing this great method, but I cant down the input_file and checkpoint from your link, can you share a google cloud or some other source link?

Hi,@ParthaEth,when I run the generate_random_samples.py,it has some problems in /GIF-master/my_utils/photometric_optimization/renderer.py,could you please tell me how to solve it?Thank you very much!

I have downloaded both FLAME 2020 and texture space from the given website, however i cannot find the obj file from any of them.

cannot Download the FLAME_texture_data

Thank you for the nice work! I'm trying to run the pre-trained model but failed.How can I get FLAME_texture.npz and head_template_mesh.obj?

when I run python train.py --run_id 0, an error appear:

Dear authors, thank you for sharing your valuable work. I was trying to generate the results like table 3 of the paper using the script GIF/tests/deca_inf_vs_given_cond_landmark_viz.py. Here, I could not find the paths for flame_datapath and face_region_only_indices variable.

File "role_of_different_parameters.py", line 108, in
flame_decoder = overlay_visualizer.deca.flame.eval()
AttributeError: 'OverLayViz' object has no attribute 'deca'

Hi, thanks for your great work!
I have completed the configuration followed first thing first section of the read me,
But when I run python generate_random_samples.py , I got some errors, Can you give me some proposal！thanks!

../GIF-master/my_utils/photometric_optimization/renderer.py line 94
verts, faces, aux = load_obj(obj_filename) #line:93
uvcoords = aux.verts_uvs[None, ...] # (N, V, 2) line:94

File "../GIF-master/my_utils/photometric_optimization/renderer.py", line 94, in init
TypeError: 'NoneType' object is not subscriptable

I not sure it is model version or code version issue?

File "generate_random_samples.py", line 162, in
flm_batch = position_to_given_location(flame_decoder, flm_batch)
File "../my_utils/eye_centering.py", line 39, in position_to_given_location
verts, _, _ = deca_flame_decoder(shape_params=shape, expression_params=expression, pose_params=pose)
File "/home/ubuntu/.conda/envs/gif/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "../my_utils/photometric_optimization/models/FLAME.py", line 204, in forward
self.lbs_weights, dtype=self.dtype)
File "../my_utils/photometric_optimization/models/lbs.py", line 211, in lbs
J_transformed, A = batch_rigid_transform(rot_mats, J, parents, dtype=dtype)
File "../my_utils/photometric_optimization/models/lbs.py", line 353, in batch_rigid_transform
rel_joints.view(-1, 3, 1)).view(-1, joints.shape[1], 4, 4)
RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.

Thanks to your work "GIF: Generative Interpretable Faces", and I'm interested in this work and desiring for the source code. I wonder when it will be public, or can you send the original code to me? Thanks, looking forward your reply.

Hi @ParthaEth,thanks for your great work, and I have try to train on my own dataset based on your pretrained model. while the generated images look fine , but the saved meshes in the conditon dir are all black.

Hope you can give some advices, thanks~

Hi, @ParthaEth, I have followed your step to create my own datasets, but after train for a few days, the test results are all bad(I only use one person(6k images) to train, and it contain abundant expression and pose )

I first use DECA to get parameters and then create the datasets use the scripts in prepare_lmdb
My loss values are nearly like below: G_loss=5.x/ D_loss=1.x/embd_reg_loss=0/interp_loss=2.x

Hope you can give some advice, thanks~

I have got this issue when I try to run python generate_voca_animation.py

any hint?

Output:
Collating FFHQ parameters
Collating FFHQ parameters, done!
<<<<<<<<<< Running Style GAN 2 >>>>>>>>>>>>>>>>>>>>>>>
generator const_input n_params: 8192
generator to_rgb n_params: 1568667
generator progression n_params: 27955884
generator z_to_w n_params: 2101248
creating the FLAME Decoder
/home/wiam/Desktop/GIF/my_utils/photometric_optimization/models/FLAME.py:92: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
self.register_buffer('dynamic_lmk_faces_idx', torch.tensor(lmk_embeddings['dynamic_lmk_faces_idx'], dtype=torch.long))
/home/wiam/Desktop/GIF/my_utils/photometric_optimization/models/FLAME.py:93: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
self.register_buffer('dynamic_lmk_bary_coords', torch.tensor(lmk_embeddings['dynamic_lmk_bary_coords'], dtype=self.dtype))
/home/wiam/anaconda3/envs/gif3/lib/python3.8/site-packages/pytorch3d/io/obj_io.py:533: UserWarning: Mtl file does not exist: /home/wiam/Desktop/GIF/GIF_resources/input_files//flame_resource/template.mtl
warnings.warn(f"Mtl file does not exist: {f}")
creating the FLAME Decoder
/home/wiam/Desktop/GIF/my_utils/photometric_optimization/models/FLAME.py:92: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
self.register_buffer('dynamic_lmk_faces_idx', torch.tensor(lmk_embeddings['dynamic_lmk_faces_idx'], dtype=torch.long))
/home/wiam/Desktop/GIF/my_utils/photometric_optimization/models/FLAME.py:93: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
self.register_buffer('dynamic_lmk_bary_coords', torch.tensor(lmk_embeddings['dynamic_lmk_bary_coords'], dtype=self.dtype))
0%| | 0/30 [00:00<?, ?it/s]/home/wiam/anaconda3/envs/gif3/lib/python3.8/site-packages/torch/nn/functional.py:3060: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
warnings.warn("Default upsampling behavior when mode={} is changed "
0%| | 0/30 [00:26<?, ?it/s]
Traceback (most recent call last):
File "generate_voca_animation.py", line 90, in
overlay_visualizer.get_rendered_mesh(flame_params=(shape_batch, exp_batch, pose_batch,
File "/home/wiam/Desktop/GIF/my_utils/visualize_flame_overlay.py", line 24, in get_rendered_mesh
self.rendering_helper.render_tex_and_normal(shapecode=shape, expcode=expression,
File "/home/wiam/Desktop/GIF/my_utils/photometric_optimization/gif_helper.py", line 33, in render_tex_and_normal
albedos = self.flametex(texcode)
File "/home/wiam/anaconda3/envs/gif3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(input, **kwargs)
File "/home/wiam/Desktop/GIF/my_utils/photometric_optimization/models/FLAME.py", line 256, in forward
texture = self.texture_mean + (self.texture_basistexcode[:,None,:]).sum(-1)
RuntimeError: CUDA out of memory. Tried to allocate 4.69 GiB (GPU 0; 11.17 GiB total capacity; 1.63 GiB already allocated; 4.20 GiB free; 6.41 GiB reserved in total by PyTorch)

Thank for your work!
I can't find "texture_data_FLAME_256.npy"
I only have "FLAME_texture.npz". Does "texture_data_FLAME_256.npy" come from FLAME_texture.npz?

Thanks for your work.
When I run codes under "plots/voca/", I got an error: Can't find the file named "b_box_stats.npz", I found that the path of "b_box_stats.npz" is from "constants.py",line 50.
I don’t know where to get this file. Could you tell me how to get it?