wuhaozhe / pytorch-nicp Goto Github PK

Hi, with landmarks: landmarks = torch.from_numpy(np.array(landmarks)).to(device).long(), maybe you can reshape landmarks from torch.Size([1, 1, 68, 2]) to torch.Size([1, 68, 2])

Originally posted by @wuhaozhe in #3 (comment)
hi！I got output as torch.Size([1, 68, 512, 3]) torch.Size([1, 68, 2]) torch.Size([1, 512, 512, 3])
I think the shape of following tensors are right, but I meet the same problem.
lm_vertex = torch.gather(lm_vertex, 2, column_index)
RuntimeError: CUDA error: device-side assert triggered

landmarks = torch.from_numpy(np.array(landmarks)).to(device).long()

row_index = landmarks[:, :, 1].view(landmarks.shape[0], -1)
column_index = landmarks[:, :, 0].view(landmarks.shape[0], -1)
row_index = row_index.unsqueeze(2).unsqueeze(3).expand(landmarks.shape[0], landmarks.shape[1], shape_img.shape[2], shape_img.shape[3])
column_index = column_index.unsqueeze(1).unsqueeze(3).expand(landmarks.shape[0], landmarks.shape[1], landmarks.shape[1], shape_img.shape[3])
print(row_index.shape, landmarks.shape, shape_img.shape)

https://github.com/wuhaozhe/pytorch-nicp/blob/099f189b749154e97d0f6fa5a1e16e8fb885cce0/bfm_model.py#L23C60-L23C60

lm_index = bfm_meta_data['keypoints']
faces = torch.from_numpy(faces).long().to(device) - 1
lm_index = torch.from_numpy(lm_index).long().to(device)

According to this line from 3dmm-fitting-pytorch repo, self.kp_inds = torch.tensor(model_dict['keypoints'] - 1).squeeze().long()](https://github.com/ascust/3DMM-Fitting-Pytorch/blob/master/core/BFM09Model.py#L26) , whether lm_index should be also minus 1 ?

Hello,

On my computer it seems quite slow to run demo_nicp.py. At least it took more than 1 minutes to get final.obj. Is it correct?

I ran AMM_NRR for non-rigit ICP registration with two 7000 vertices meshes. It needs ca 1 second with CPU on my computer. With GPU, it might be possible to do the same work in less than 100 ms?

Thank you!

Traceback (most recent call last):
File "demo_nicp.py", line 28, in
bfm_meshes, bfm_lm_index = load_bfm_model(torch.device('cuda:0'))
File "/data/pytorch-nicp/bfm_model.py", line 15, in load_bfm_model
bfm_meta_data = loadmat('BFM/BFM09_model_info.mat')
File "/root/anaconda3/envs/pytorch3d/lib/python3.8/site-packages/scipy/io/matlab/mio.py", line 224, in loadmat
with _open_file_context(file_name, appendmat) as f:
File "/root/anaconda3/envs/pytorch3d/lib/python3.8/contextlib.py", line 113, in enter
return next(self.gen)
File "/root/anaconda3/envs/pytorch3d/lib/python3.8/site-packages/scipy/io/matlab/mio.py", line 17, in _open_file_context
f, opened = _open_file(file_like, appendmat, mode)
File "/root/anaconda3/envs/pytorch3d/lib/python3.8/site-packages/scipy/io/matlab/mio.py", line 45, in _open_file
return open(file_like, mode), True
FileNotFoundError: [Errno 2] No such file or directory: 'BFM/BFM09_model_info.mat'

In 3DMMfitting-pytorch， there are only these files:
BFM_exp_idx.mat
BFM_front_idx.mat
facemodel_info.mat
README.md
select_vertex_id.mat
similarity_Lm3D_all.mat
std_exp.txt

Traceback (most recent call last):
File "demo_nicp.py", line 27, in
target_lm_index, lm_mask = get_mesh_landmark(norm_meshes, dummy_render)
File "/data/pytorch-nicp/landmark.py", line 37, in get_mesh_landmark
row_index = row_index.unsqueeze(2).unsqueeze(3).expand(landmarks.shape[0], landmarks.shape[1], shape_img.shape[2], shape_img.shape[3])
RuntimeError: The expanded size of the tensor (1) must match the existing size (2) at non-singleton dimension 1. Target sizes: [1, 1, 512, 3]. Tensor sizes: [1, 2, 1, 1]

I have already configure the environment，but it seems have some problems in the code.What can I do to solve this problem.