facebookresearch / meshrcnn

code for Mesh R-CNN, ICCV 2019

Mesh R-CNN

Code for the paper

Mesh R-CNN
Georgia Gkioxari, Jitendra Malik, Justin Johnson
ICCV 2019

 

(thanks to Alberto Tono!)

Installation Requirements

• Detectron2
• PyTorch3D

The implementation of Mesh R-CNN is based on Detectron2 and PyTorch3D. You will first need to install those in order to be able to run Mesh R-CNN.

To install

git clone https://github.com/facebookresearch/meshrcnn.git
cd meshrcnn && pip install -e .

Demo

Run Mesh R-CNN on an input image

python demo/demo.py \
--config-file configs/pix3d/meshrcnn_R50_FPN.yaml \
--input /path/to/image \
--output output_demo \
--onlyhighest MODEL.WEIGHTS meshrcnn://meshrcnn_R50.pth

See demo.py for more details.

Running Experiments

Pix3D

See INSTRUCTIONS_PIX3D.md for more instructions.

ShapeNet

See INSTRUCTIONS_SHAPENET.md for more instructions.

License

The Mesh R-CNN codebase is released under BSD-3-Clause License

meshrcnn's Issues

Difficulty running demo.py on mac

Hi, I followed the setup process for meshrcnn and detctron2

I am trying to run the demo script with this command on MacOS

python demo.py
--config-file /Users/nk/Downloads/Michigan/Computer-Science/Code/meshrcnn/configs/pix3d/meshrcnn_R50_FPN.yaml
--input /Users/nk/Downloads/sampleimage.HEIC
--output /Users/nk/Downloads
--onlyhighest MODEL.WEIGHTS meshrcnn://meshrcnn_R50.pth \

my Torch device in the demo script is set to CPU

Do you have any advice?

Question about voxels in training data

Hi,

first, thank you for sharing your code!

I have a question about the need for voxelized meshes in the training data. If I understand this correctly, the voxels in training data are used to minimize the voxel prediction loss. And the shapenet training actually relies on the precalculated model.binvox files.

But the Pix3d training set does not have the model.binvox files... What happens then in case of pix3d - does the voxel head not get trained? or voxelization of the training Pix3D meshes happens on the fly?

The training procedure

Hi,
I tried to train the network with data from pix3d. I saw the training metrics showing up during the training. What about validation? We want to use validation data to monitor the training procedure to prevent overfitting, right? But it seems that you only did the test on the test dataset at the end of the training.
Best regards,
zshyang.

Preprocessing ShapeNet

I downloaded ShapeNetCore.v1.zip and ShapeNetCore.v1.binvox.zip. Then I unzipped them and run preprocess_shapenet.py. Output:
Traceback (most recent call last):
File "preprocess_shapenet.py", line 317, in
main(args)
File "preprocess_shapenet.py", line 144, in main
raise ValueError("Pre processing identified missing data points")
ValueError: Pre processing identified missing data points

Can you help me?

Any idea about how to fix the following bug?

It seems that the box loss became NaN at some point.

FloatingPointError: Loss became infinite or NaN at iteration=756!
loss_dict = {'loss_cls': tensor(nan, device='cuda:1', grad_fn=<NllLossBackward>), 
'loss_box_reg': tensor(nan, device='cuda:1', grad_fn=<DivBackward0>), 'loss_z_reg': 
tensor(nan, device='cuda:1', grad_fn=<DivBackward0>), 'loss_mask': tensor(nan, 
device='cuda:1', grad_fn=<BinaryCrossEntropyWithLogitsBackward>), 'loss_voxel': tensor(nan, 
device='cuda:1', grad_fn=<MulBackward0>), 'loss_chamfer': tensor(-0., device='cuda:1', 
grad_fn=<MulBackward0>), 'loss_normals': tensor(-0., device='cuda:1', grad_fn=
<MulBackward0>), 'loss_edge': tensor(-0., device='cuda:1', grad_fn=<MulBackward0>), 
'loss_rpn_cls': tensor(nan, device='cuda:1', grad_fn=<MulBackward0>), 'loss_rpn_loc': 
tensor(inf, device='cuda:1', grad_fn=<MulBackward0>)}

Failed to install meshrcnn ...

It's saying Successfully installed meshrcnn. However, by checking the following message:

 36   master  ✔  pip install --no-deps -e .
Defaulting to user installation because normal site-packages is not writeable
Obtaining file://....../meshrcnn
Installing collected packages: meshrcnn
  Attempting uninstall: meshrcnn
    Found existing installation: meshrcnn 1.0
    Uninstalling meshrcnn-1.0:
      Successfully uninstalled meshrcnn-1.0
  Running setup.py develop for meshrcnn
Successfully installed meshrcnn
Traceback (most recent call last):
  File "~/.local/bin/pip", line 8, in <module>
    sys.exit(main())
  File "~/.local/lib/python3.6/site-packages/pip/_internal/cli/main.py", line 75, in main
    return command.main(cmd_args)
  File "~/.local/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 114, in main
    return self._main(args)
  File "~/.local/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 226, in _main
    self.handle_pip_version_check(options)
  File "~/.local/lib/python3.6/site-packages/pip/_internal/cli/req_command.py", line 155, in handle_pip_version_check
    timeout=min(5, options.timeout)
  File "~/.local/lib/python3.6/site-packages/pip/_internal/cli/req_command.py", line 100, in _build_session
    index_urls=self._get_index_urls(options),
  File "~/.local/lib/python3.6/site-packages/pip/_internal/network/session.py", line 249, in __init__
    self.headers["User-Agent"] = user_agent()
  File "~/.local/lib/python3.6/site-packages/pip/_internal/network/session.py", line 159, in user_agent
    setuptools_version = get_installed_version("setuptools")
  File "~/.local/lib/python3.6/site-packages/pip/_internal/utils/misc.py", line 634, in get_installed_version
    working_set = pkg_resources.WorkingSet()
  File "~/.local/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 567, in __init__
    self.add_entry(entry)
  File "~/.local/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 623, in add_entry
    for dist in find_distributions(entry, True):
  File "~/.local/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1983, in find_eggs_in_zip
    if metadata.has_metadata('PKG-INFO'):
  File "~/.local/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1414, in has_metadata
    return self._has(path)
  File "~/.local/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1854, in _has
    return zip_path in self.zipinfo or zip_path in self._index()
  File "~/.local/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1731, in zipinfo
    return self._zip_manifests.load(self.loader.archive)
  File "~/.local/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1688, in load
    mtime = os.stat(path).st_mtime
FileNotFoundError: [Errno 2] No such file or directory: '~/.local/lib/python3.6/site-packages/meshrcnn-1.0-py3.6.egg'

and I doublchecked it's true that there is NO such file ~/.local/lib/python3.6/site-packages/meshrcnn-1.0-py3.6.egg. Aftewards, I found there is an egg find produce under current subfolder dist. I just did a copy successfully.

 57   master  ✘  ll ~/.local/lib/python3.6/site-packages/meshrcnn-1.0-py3.6.egg
ls: cannot access '~/.local/lib/python3.6/site-packages/meshrcnn-1.0-py3.6.egg': No such file or directory
 64   master  ✔  ls dist/
meshrcnn-1.0-py3.6.egg*
 65   master  ✔  cp dist/meshrcnn-1.0-py3.6.egg ~/.local/lib/python3.6/site-packages/   

Anyway, after that, I ended up with a failure demo as:

 66   master  ✔  python demo/demo.py \
                     --config-file configs/pix3d/meshrcnn_R50_FPN.yaml \
                     --input ../Mesh-R-CNN.jpg \
                     --output ../output_demo \
                     --onlyhighest MODEL.WEIGHTS ../models/meshrcnn_R50.pth
Traceback (most recent call last):
  File "demo/demo.py", line 11, in <module>
    from detectron2.data import MetadataCatalog
  File "~/.local/lib/python3.6/site-packages/detectron2-0.1.3-py3.6-linux-x86_64.egg/detectron2/data/__init__.py", line 4, in <module>
    from .build import (
  File "~/.local/lib/python3.6/site-packages/detectron2-0.1.3-py3.6-linux-x86_64.egg/detectron2/data/build.py", line 14, in <module>
    from detectron2.structures import BoxMode
  File "~/.local/lib/python3.6/site-packages/detectron2-0.1.3-py3.6-linux-x86_64.egg/detectron2/structures/__init__.py", line 6, in <module>
    from .keypoints import Keypoints, heatmaps_to_keypoints
  File "~/.local/lib/python3.6/site-packages/detectron2-0.1.3-py3.6-linux-x86_64.egg/detectron2/structures/keypoints.py", line 6, in <module>
    from detectron2.layers import interpolate
  File "~/.local/lib/python3.6/site-packages/detectron2-0.1.3-py3.6-linux-x86_64.egg/detectron2/layers/__init__.py", line 3, in <module>
    from .deform_conv import DeformConv, ModulatedDeformConv
  File "~/.local/lib/python3.6/site-packages/detectron2-0.1.3-py3.6-linux-x86_64.egg/detectron2/layers/deform_conv.py", line 10, in <module>
    from detectron2 import _C
ImportError: ~/.local/lib/python3.6/site-packages/detectron2-0.1.3-py3.6-linux-x86_64.egg/detectron2/_C.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN6caffe28TypeMeta21_typeMetaDataInstanceISt7complexIfEEEPKNS_6detail12TypeMetaDataEv

Tranformation applied on the voxels.

I was wondering if there is any transformation applied to the meshes before voxelizing them, because I tried to regenerate the .mat files from the .obj files provided in the pix3d Dataset, This is related to

model : "model/bed/IKEA_BEDDINGE/model.obj"
and its corresponding
voxel : "model/bed/IKEA_BEDDINGE/voxel.obj"

Here are the voxels visualized,

The Images on the left are the voxels from the dataset provided in the dataset and the Images on right are the ones I have created from binvox, the provided mesh model(.obj) already is transformed in the translation [0, 0, 0] and rotation [pie/2, 0, 0](XYZ Euler), I had to rotate the model to [pie, 0, 0] to get the correct orientation shown in above Image.

But I can see that there is some translation present in the voxels. So are any transformations applied to the voxels?

Issue on running demo script

Hi , thanks for sharing this fantastic library. I was trying to run the demo script using an image taken from the pix3d dataset.

However I get this error. Apologies if I'm doing something silly here, I tried to find the issue but have been struggling

` $ python demo/demo.py --config-file configs/pix3d/meshrcnn_R50_FPN.yaml --input "0041.png" --output output_demo --onlyhighest MODEL.WEIGHTS meshrcnn://meshrcnn_R50.pth


[02/16 15:54:30 demo]: Arguments: Namespace(config_file='configs/pix3d/meshrcnn_R50_FPN.yaml', focal_length=20.0, input='0041.png', onlyhighest=True, opts=['MODEL.WEIGHTS', 'meshrcnn://meshrcnn_R50.pth'], output='output_demo')

Traceback (most recent call last):
  File "demo/demo.py", line 227, in <module>
    predictions = demo.run_on_image(img, focal_length=args.focal_length)
  File "demo/demo.py", line 59, in run_on_image
    predictions = self.predictor(image)
  File "/home/anthony/miniconda3/envs/meshRCNN/lib/python3.6/site-packages/detectron2/engine/defaults.py", line 196, in __call__
    predictions = self.model([inputs])[0]
  File "/home/anthony/miniconda3/envs/meshRCNN/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/anthony/miniconda3/envs/meshRCNN/lib/python3.6/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 107, in forward
    return self.inference(batched_inputs)
  File "/home/anthony/miniconda3/envs/meshRCNN/lib/python3.6/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 169, in inference
    results, _ = self.roi_heads(images, features, proposals, None)
  File "/home/anthony/miniconda3/envs/meshRCNN/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/anthony/repos/meshrcnn/meshrcnn/modeling/roi_heads/roi_heads.py", line 181, in forward
    pred_instances = self.forward_with_given_boxes(features, pred_instances)
  File "/home/anthony/repos/meshrcnn/meshrcnn/modeling/roi_heads/roi_heads.py", line 202, in forward_with_given_boxes
    instances = self._forward_mask(features, instances)
  File "/home/anthony/repos/meshrcnn/meshrcnn/modeling/roi_heads/roi_heads.py", line 267, in _forward_mask
    return self.mask_head(mask_features, instances)
  File "/home/anthony/miniconda3/envs/meshRCNN/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
TypeError: forward() takes 2 positional arguments but 3 were given

```


`

RuntimeError: CUDA out of memory. Tried to allocate 442.00 MiB (GPU 0; 15.90 GiB total capacity; 14.37 GiB already allocated; 193.88 MiB free; 14.92 GiB reserved in total by PyTorch) while trying to train using trainer.train()

I am trying to train meshrcnn on the Pix3D dataset in Google Colab using GPU backend and 25GB RAM space. I am trying to implement it myself without explicitly calling the /tools/train_net.py file.

So, I first created a Namespace type object having the required arguments:
args = Namespace(config_file='configs/pix3d/meshrcnn_R50_FPN.yaml', dist_url='tcp://127.0.0.1:49152', eval_only=False, machine_rank=0, num_gpus=8, num_machines=1, opts=[], resume=False)
Now, I changed the MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE to 8 in place of 64 in 'configs/pix3d/meshrcnn_R50_FPN.yaml' file and used it so that Colab does not run out of memory)

I then created a cfg variable and included all the necessary keys and default values:
cfg = get_cfg()
get_meshrcnn_cfg_defaults(cfg) ##get_meshrcnn_cfg_defaults imported from meshrcnn
cfg.merge_from_file(args.config_file)
cfg.merge_from_list(args.opts)
cfg.freeze()
default_setup(cfg, args) ##default_setup imported from detectron2

Then I created the trainer object and tried to train:
trainer = Trainer(cfg)
trainer.resume_or_load(resume=args.resume)
trainer.train()

But I am getting "RuntimeError: CUDA out of memory. Tried to allocate 442.00 MiB (GPU 0; 15.90 GiB total capacity; 14.37 GiB already allocated; 193.88 MiB free; 14.92 GiB reserved in total by PyTorch)"
I double checked that GPU was NOT utilised 100% from the bar above which shows the utlised RAM. Only 5.27GB/25GB was utilised.

Ways to reproduce my code:
This is my Google Colab Notebook link

Is there any way (like, changing some other parameter values or something) by which I can train the meshrcnn model on Pix3D dataset in Google Colab?

@gkioxari kindly look into it.
pytorch3d version is '0.2.0'
torchvision version is '0.6.0+cu101'

I also checked for similar type of issues in pytorch forums but could not solve this problem.

The Error:
[06/06 04:39:45 d2.engine.train_loop]: Starting training from iteration 0
ERROR [06/06 04:39:46 d2.engine.train_loop]: Exception during training:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/detectron2/engine/train_loop.py", line 132, in train
self.run_step()
File "/usr/local/lib/python3.6/dist-packages/detectron2/engine/train_loop.py", line 215, in run_step
loss_dict = self.model(data)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/detectron2/modeling/meta_arch/rcnn.py", line 121, in forward
features = self.backbone(images.tensor)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/detectron2/modeling/backbone/fpn.py", line 136, in forward
results.insert(0, output_conv(prev_features))
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/detectron2/layers/wrappers.py", line 93, in forward
x = super().forward(x)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/conv.py", line 349, in forward
return self._conv_forward(input, self.weight)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/conv.py", line 346, in _conv_forward
self.padding, self.dilation, self.groups)
RuntimeError: CUDA out of memory. Tried to allocate 442.00 MiB (GPU 0; 15.90 GiB total capacity; 14.37 GiB already allocated; 193.88 MiB free; 14.92 GiB reserved in total by PyTorch)
[06/06 04:39:46 d2.engine.hooks]: Total training time: 0:00:01 (0:00:00 on hooks)

RuntimeError Traceback (most recent call last)
in ()
1 #This is where we are having errors
----> 2 trainer.train()

10 frames
/usr/local/lib/python3.6/dist-packages/detectron2/engine/defaults.py in train(self)
399 OrderedDict of results, if evaluation is enabled. Otherwise None.
400 """
--> 401 super().train(self.start_iter, self.max_iter)
402 if len(self.cfg.TEST.EXPECTED_RESULTS) and comm.is_main_process():
403 assert hasattr(

/usr/local/lib/python3.6/dist-packages/detectron2/engine/train_loop.py in train(self, start_iter, max_iter)
130 for self.iter in range(start_iter, max_iter):
131 self.before_step()
--> 132 self.run_step()
133 self.after_step()
134 except Exception:

/usr/local/lib/python3.6/dist-packages/detectron2/engine/train_loop.py in run_step(self)
213 If you want to do something with the losses, you can wrap the model.
214 """
--> 215 loss_dict = self.model(data)
216 losses = sum(loss_dict.values())
217 self._detect_anomaly(losses, loss_dict)

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in call(self, *input, **kwargs)
548 result = self._slow_forward(*input, **kwargs)
549 else:
--> 550 result = self.forward(*input, **kwargs)
551 for hook in self._forward_hooks.values():
552 hook_result = hook(self, input, result)

/usr/local/lib/python3.6/dist-packages/detectron2/modeling/meta_arch/rcnn.py in forward(self, batched_inputs)
119 gt_instances = None
120
--> 121 features = self.backbone(images.tensor)
122
123 if self.proposal_generator:

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in call(self, *input, **kwargs)
548 result = self._slow_forward(*input, **kwargs)
549 else:
--> 550 result = self.forward(*input, **kwargs)
551 for hook in self._forward_hooks.values():
552 hook_result = hook(self, input, result)

/usr/local/lib/python3.6/dist-packages/detectron2/modeling/backbone/fpn.py in forward(self, x)
134 if self._fuse_type == "avg":
135 prev_features /= 2
--> 136 results.insert(0, output_conv(prev_features))
137
138 if self.top_block is not None:

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in call(self, *input, **kwargs)
548 result = self._slow_forward(*input, **kwargs)
549 else:
--> 550 result = self.forward(*input, **kwargs)
551 for hook in self._forward_hooks.values():
552 hook_result = hook(self, input, result)

/usr/local/lib/python3.6/dist-packages/detectron2/layers/wrappers.py in forward(self, x)
91 return empty
92
---> 93 x = super().forward(x)
94 if self.norm is not None:
95 x = self.norm(x)

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/conv.py in forward(self, input)
347
348 def forward(self, input):
--> 349 return self._conv_forward(input, self.weight)
350
351 class Conv3d(_ConvNd):

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/conv.py in _conv_forward(self, input, weight)
344 _pair(0), self.dilation, self.groups)
345 return F.conv2d(input, weight, self.bias, self.stride,
--> 346 self.padding, self.dilation, self.groups)
347
348 def forward(self, input):

RuntimeError: CUDA out of memory. Tried to allocate 442.00 MiB (GPU 0; 15.90 GiB total capacity; 14.37 GiB already allocated; 193.88 MiB free; 14.92 GiB reserved in total by PyTorch)

Besides that. I also tried using:
python tools/train_net.py --num-gpus 1 \
--config-file configs/pix3d/meshrcnn_R50_FPN.yaml
That also gave same type of error:
RuntimeError: CUDA out of memory

Issue with installation: RuntimeError: Not compiled with GPU support

Hello I tried running the demo file after installation of meshrcnn. This is the error I get:

python demo/demo.py --config-file configs/pix3d/meshrcnn_R50_FPN.yaml --input chic-open-floor-plan.png --o utput output_demo --onlyhighest MODEL.WEIGHTS meshrcnn://meshrcnn_R50.pth [06/24 23:35:08 demo]: Arguments: Namespace(config_file='configs/pix3d/meshrcnn_R50_FPN.yaml', focal_length=20.0, input='chic-open-floor-plan.png ', onlyhighest=True, opts=['MODEL.WEIGHTS', 'meshrcnn://meshrcnn_R50.pth'], output='output_demo') /opt/conda/conda-bld/pytorch_1591914858187/work/torch/csrc/utils/python_arg_parser.cpp:756: UserWarning: This overload of nonzero is deprecated: nonzero(Tensor input, *, Tensor out) Consider using one of the following signatures instead: nonzero(Tensor input, *, bool as_tuple) /opt/conda/conda-bld/pytorch_1591914858187/work/aten/src/ATen/native/BinaryOps.cpp:81: UserWarning: Integer division of tensors using div or / is deprecated, and in a future release div will perform true division as in Python 3. Use true_divide or floor_divide (// in Python) instead. Traceback (most recent call last): File "demo/demo.py", line 227, in <module> predictions = demo.run_on_image(img, focal_length=args.focal_length) File "demo/demo.py", line 59, in run_on_image predictions = self.predictor(image)
I am working on cuda 10.2. I have installed pytorch3d using conda and the prebuilt library of detectron2 for cuda 10.2 and 1.5. I have looked at the previous issue #27 on related issue but i think I am installing everything correctly. Any help will be really appreciated
The conda list is in the image attached:

Do not sure about what is the output.

Hi,
Is the output mesh from the network is in camera space? Or in object space?
For camera space, I mean the output mesh's vertex (x, y, z) corresponds to the pixel position.
For object space, It means the object's canonical position.

mapping output to input

Not sure if this is a issue. But is it possible to map the output mesh triangles to the image pixels? On reading the paper it says there exist a natural mapping. But can we get this from the model somehow?
Thanks is much help

question about batch_crop_voxels_within_box

Thanks for sharing the great code!

I am trying to train meshrcnn on ScanNet, where each image has more instances than Pix3D.

I found this line of code tries to stack a list of voxels into (batch_size, n, 3)

meshrcnn/meshrcnn/structures/voxel.py

Line 23 in 32dfa7f

voxels_tensor = torch.stack(voxels.data, 0)

how do you make sure that each instance in the list has the same amount of voxels so that they can be stacked? or in the training, it is assumed that there is only 1 instance in the image?

Questions on Voxel head an V count

Hi,

I have a couple questions about the voxel head and the number of vertices in the final mesh.

1. In the paper you say "On ShapeNet, the Voxel-Only baseline is trained with a batch size of 64 (vs. a batch size of 32 for our full model); on Pix3D it uses the same training recipe as our full model." But looking at https://github.com/facebookresearch/meshrcnn/blob/master/configs/pix3d/MESH-RCNN-FPN.yaml#L51 it seems like the batch size was 16 for Pix3D. Am I reading it wrong?

2. Just right before that, in the paper, you say "The voxel branch is the same as in the ShapeNet models (see Table 8), except that we predict voxels at a lower resolution (48×48×48 for ShapeNet vs. 24×24×24 for Pix3D) due to memory constraints.". If I understand this correctly, the max number of vertices of the final mesh is limited by the number of voxels * 8. Is this correct, and there is no densification along the edges happening (I didn't find it in the paper, sorry if I missed it) between the mesh refinement steps?

3. If the max vertex count is truly limited by ROI_VOXEL_HEAD.NUM_DEPTH * 3 * 8, do you think that the quality metrics could potentially go higher than in the paper for Pix3D tests, providing one would train with a smaller batch size, but higher ROI_VOXEL_HEAD.NUM_DEPTH given the vRAM constraints?

undefined symbol: _ZN3c107Warning4warnENS_14SourceLocationERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE

 3   master  ✔  python demo/demo.py \ 
                    --config-file configs/pix3d/meshrcnn_R50_FPN.yaml \
                    --input /path/to/image \
                    --output output_demo \
                    --onlyhighest MODEL.WEIGHTS meshrcnn://meshrcnn_R50.pth
Traceback (most recent call last):
  File "demo/demo.py", line 11, in <module>
    from detectron2.data import MetadataCatalog
  File "~/.local/lib/python3.6/site-packages/detectron2-0.1.3-py3.6-linux-x86_64.egg/detectron2/data/__init__.py", line 4, in <module>
    from .build import (
  File "~/.local/lib/python3.6/site-packages/detectron2-0.1.3-py3.6-linux-x86_64.egg/detectron2/data/build.py", line 14, in <module>
    from detectron2.structures import BoxMode
  File "~/.local/lib/python3.6/site-packages/detectron2-0.1.3-py3.6-linux-x86_64.egg/detectron2/structures/__init__.py", line 6, in <module>
    from .keypoints import Keypoints, heatmaps_to_keypoints
  File "~/.local/lib/python3.6/site-packages/detectron2-0.1.3-py3.6-linux-x86_64.egg/detectron2/structures/keypoints.py", line 6, in <module>
    from detectron2.layers import interpolate
  File "~/.local/lib/python3.6/site-packages/detectron2-0.1.3-py3.6-linux-x86_64.egg/detectron2/layers/__init__.py", line 3, in <module>
    from .deform_conv import DeformConv, ModulatedDeformConv
  File "~/.local/lib/python3.6/site-packages/detectron2-0.1.3-py3.6-linux-x86_64.egg/detectron2/layers/deform_conv.py", line 10, in <module>
    from detectron2 import _C
ImportError: ~/.local/lib/python3.6/site-packages/detectron2-0.1.3-py3.6-linux-x86_64.egg/detectron2/_C.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN3c107Warning4warnENS_14SourceLocationERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE

Demo in ShapeNet

Thank you for providing good code.
I want to check 3D reconstruction Object in ShapeNet like Demo.
How can i ?

Invalid multinomial distribution for poor predictions

Hi, @gkioxari,
I have a question regarding the compare_meshes function used in pix3d_evaluation.py. I tried this function on a new dataset and got some poor predictions during evaluation for various reasons. I understand if this is out of the scope of issues. I just want to see if you can give me some advice on how to avoid getting errors like this during evaluation. Any suggestions would be much appreciated!

During the evaluation, I first got the following error:

...
../pytorch3d/ops/sample_points_from_meshes.py line 71
sample_face_idxs += mesh_to_face[meshes.valid].view(num_valid_meshes, 1)
RuntimeError: copy_if failed to synchronize: device-side assert triggered"

Then, I saved all meshes and called compare_meshes in a notebook. I found that for some meshes, such as this pred and gt pair, I got "invalid multinomial distribution (sum of probabilities <= 0)" error, which might be causing the error above. The full bt looks like this:

----> 9         shape_metrics = compare_meshes(pred_meshes, gt_meshes, reduce=False)

.../python3.6/site-packages/torch/autograd/grad_mode.py in decorate_no_grad(*args, **kwargs)
     47         def decorate_no_grad(*args, **kwargs):
     48             with self:
---> 49                 return func(*args, **kwargs)
     50         return decorate_no_grad
     51 

.../meshrcnn/meshrcnn/utils/metrics.py in compare_meshes(pred_meshes, gt_meshes, num_samples, scale, thresholds, reduce, eps)
     65         num_samples_pred = num_samples_gt = num_samples
     66 
---> 67     pred_points, pred_normals = _sample_meshes(pred_meshes, num_samples_pred)
     68     gt_points, gt_normals = _sample_meshes(gt_meshes, num_samples_gt)
     69     if pred_points is None:

.../meshrcnn/meshrcnn/utils/metrics.py in _sample_meshes(meshes, num_samples)
    154             verts = meshes.verts_list
    155     else:
--> 156         verts, normals = sample_points_from_meshes(meshes, num_samples, return_normals=True)
    157     return verts, normals
    158 

.../pytorch3d/pytorch3d/ops/sample_points_from_meshes.py in sample_points_from_meshes(meshes, num_samples, return_normals)
     63         sample_face_idxs = areas_padded.multinomial(
     64             num_samples, replacement=True
---> 65         )  # (N, num_samples)
     66 
     67         # print("meshes.valid", meshes.valid)

RuntimeError: invalid multinomial distribution (sum of probabilities <= 0)

Here is a snapshot of the first couple of vertices in the predicted obj file. As you can see there are a lot of repeated vertices.

v -1.661227 1.381543 3.834221
v -1.661227 1.381543 3.834221
v -1.661227 1.381543 3.834221
v -1.661227 1.381543 3.834221
v -1.661227 1.381543 3.834222
v -1.661227 1.381543 3.834222
v -1.661227 1.381543 3.834221
v -1.661227 1.381543 3.834221
v -1.661227 1.381543 3.834221
v -1.661227 1.381543 3.834222
v -1.661227 1.381543 3.834221
v -1.661227 1.381543 3.834222
v -1.661227 1.381543 3.834221
v -1.661227 1.381543 3.834222
v -1.661227 1.381543 3.834221
v -1.661227 1.381543 3.834221
v -1.661227 1.381543 3.834221
v -1.661227 1.381543 3.834222
v -1.661227 1.381543 3.834221
v -1.661227 1.381543 3.834221
v -1.661227 1.381543 3.834221
v -1.661227 1.381543 3.834221
v -1.661227 1.381543 3.834222
v -1.661227 1.381543 3.834221
v -1.661227 1.381543 3.834221

I was wondering if you would have any suggestions on how to avoid errors like this. My apologies if I post it on the wrong place. Thanks in advance!

how can I replace "Depth Extent Prediction" with input value

I wonder how can I use top view, side view, and front view to configure image to 3D modeling.
the idea is:
user inputs perspective view 2d image
model contains top, side, front view that guide meshrcnn to convert perspective view image to 3d model (more accurate) ?
I am trying to get more accurate 3d model from image by feeding guiding information like depth of each vertex, normal direction, mesh edge connection.

thank you in advance

Any visualization tool to visualize the training loss?

As we could see during training there are many prints out.
But without plot the loss curve it is hard to say what happened.
So do you have any plot tool for this purpose?

list index out of range

Hi, I have just updated my pytorch3d, detectron2 and meshrcnn to the newest version this morning with the following configuration:

Ubuntu 16.04
python 3.8.2
pytorch 1.5.0
CUDA 10.2
detectron2 0.1.1
pytorch3d 0.1.1

However, when I run the code for pix3d training python tools/train_net.py --num-gpus 8 --config-file configs/pix3d/meshrcnn_R50_FPN.yaml, I got the following error:

ERROR 04/24 11:46:51 d2.engine.train_loop]: Exception during training:
Traceback (most recent call last):
  File ".../detectron2/detectron2/engine/train_loop.py", line 132, in train
    self.run_step()
  File ".../detectron2/detectron2/engine/train_loop.py", line 215, in run_step
    loss_dict = self.model(data)
  File ".../anaconda3/envs/python38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File ".../anaconda3/envs/python38/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 445, in forward
    output = self.module(*inputs[0], **kwargs[0])
  File ".../anaconda3/envs/python38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File ".../detectron2/detectron2/modeling/meta_arch/rcnn.py", line 130, in forward
    _, detector_losses = self.roi_heads(images, features, proposals, gt_instances)
  File ".../anaconda3/envs/python38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File ".../meshrcnn/meshrcnn/modeling/roi_heads/roi_heads.py", line 152, in forward
    losses.update(self._forward_z(features, proposals))
  File ".../meshrcnn/meshrcnn/modeling/roi_heads/roi_heads.py", line 198, in _forward_z
    proposals, _ = select_foreground_proposals(instances, self.num_classes)
  File ".../detectron2/detectron2/modeling/roi_heads/roi_heads.py", line 71, in select_foreground_proposals
    fg_proposals.append(proposals_per_image[fg_idxs])
  File ".../detectron2/detectron2/structures/instances.py", line 134, in __getitem__
    ret.set(k, v[item])
  File ".../meshrcnn/meshrcnn/structures/voxel.py", line 75, in __getitem__
    return VoxelInstances(selected_data)
  File ".../meshrcnn/meshrcnn/structures/voxel.py", line 52, in __init__
    assert torch.is_tensor(voxels[0])
IndexError: list index out of range

Could someone give me some advice on what might have caused this? Thanks!

no issue

no issue

Problem with anotations

Hi,
I was thinking, do you support multiple objects? I know it is not your paper's point. But in reality, there are lots of multiple objects in an image. While I have checked the Pix3D dataset, all the ground truths only provided with one mask.
So if I provide more than one mask in my own dataset. Do you support this feature?
Best regards,
zshyang

Running Demo without NVIDIA GPU

Hey,

I want to try out the demo but I dont have NVIDIA GPU on my system, when I try to run demo, by this command

python demo/demo.py --config-file configs/pix3d/voxelrcnn_R50_FPN.yaml --input MyImages/input/img1.png --output MyImages/output1 --onlyhighest MODEL.WEIGHTS meshrcnn://voxmesh_R50.pth

I am getting this error:

[02/27 09:42:23 demo]: Arguments: Namespace(config_file='configs/pix3d/voxelrcnn_R50_FPN.yaml', focal_length=20.0, input='MyImages/input/img1.png', onlyhighest=True, opts=['MODEL.WEIGHTS', 'meshrcnn://voxmesh_R50.pth'], output='MyImages/output1')
Traceback (most recent call last):
  File "demo/demo.py", line 222, in <module>
    cfg, vis_highest_scoring=args.onlyhighest, output_dir=os.path.join(args.output, im_name)
  File "demo/demo.py", line 44, in __init__
    self.predictor = DefaultPredictor(cfg)
  File "/usr/local/lib/python3.6/dist-packages/detectron2/engine/defaults.py", line 162, in __init__
    self.model = build_model(self.cfg)
  File "/usr/local/lib/python3.6/dist-packages/detectron2/modeling/meta_arch/build.py", line 19, in build_model
    return META_ARCH_REGISTRY.get(meta_arch)(cfg)
  File "/usr/local/lib/python3.6/dist-packages/detectron2/modeling/meta_arch/rcnn.py", line 41, in __init__
    pixel_mean = torch.Tensor(cfg.MODEL.PIXEL_MEAN).to(self.device).view(num_channels, 1, 1)
  File "/usr/local/lib/python3.6/dist-packages/torch/cuda/__init__.py", line 192, in _lazy_init
    _check_driver()
  File "/usr/local/lib/python3.6/dist-packages/torch/cuda/__init__.py", line 102, in _check_driver
    http://www.nvidia.com/Download/index.aspx""")
AssertionError: 
Found no NVIDIA driver on your system. Please check that you
have an NVIDIA GPU and installed a driver from
http://www.nvidia.com/Download/index.aspx

Is it necessary to to have NVIDIA GPU for demo.py? or is it necessary to have it for running the GPU? if not is there a way to run demo without GPU?

Evaluation hangs when num_workers > 0

I can execute the following line to perform evaluation on the ShapeNet dataset, but only when num_workers=0.

python tools/train_net_shapenet.py --eval-only --num-gpus 1 \
--config-file configs/shapenet/voxmesh_R50.yaml \
MODEL.CHECKPOINT shapenet://voxmesh_R50.pth

Whenever I increase the number of workers to anything >0, the process hangs and never gets past the first mini batch. It seems to get stuck right before the postprocess function of the data loader, which is never executed: https://github.com/facebookresearch/meshrcnn/blob/master/shapenet/data/mesh_vox.py#L211.

Environment details:
sys.platform linux
Python 3.6.10 |Anaconda, Inc.| (default, Jan 7 2020, 21:14:29) [GCC 7.3.0]
numpy 1.18.1
detectron2 0.1.1 @/home/user/miniconda3/envs/pytorch3d/lib/python3.6/site-packages/detectron2
detectron2 compiler GCC 7.3
detectron2 CUDA compiler 10.0
detectron2 arch flags sm_35, sm_37, sm_50, sm_52, sm_60, sm_61, sm_70, sm_75
DETECTRON2_ENV_MODULE
PyTorch 1.4.0 @/home/user/miniconda3/envs/pytorch3d/lib/python3.6/site-packages/torch
PyTorch debug build False
CUDA available True
GPU 0 GeForce GTX 1080 Ti
CUDA_HOME /z/sw/packages/cuda/10.0
NVCC Cuda compilation tools, release 10.0, V10.0.130
Pillow 6.0.0
torchvision 0.5.0 @/home/user/miniconda3/envs/pytorch3d/lib/python3.6/site-packages/torchvision
torchvision arch flags sm_35, sm_50, sm_60, sm_70, sm_75
cv2 4.2.0

---

How to download the `ShapeNet` dataset?

Hi,
As mentioned in your instruction,

You also need the original ShapeNet Core v1 & binvox dataset, which require registration before downloading.

I am not sure where to download the binvox dataset. Could you have me with it?

How to use the mesh to generate the mask.

Hi,
I was trying to use my own dataset. So I need to verify the generated training data, like the rotation and translation matrix, are correct.
So I used the Pix3D dataset and tried to generate the mask using the ground truth mesh, the rotation matrix, the translation matrix, and the camera matrix. But I could not generate the wanted mask here.
The brief code is like this.

# Define names.
obj_file_name = record["annotations"][0]['mesh']
image_path = record["file_name"]
R = record["annotations"][0]["R"] 
t = record["annotations"][0]["t"] 
K = np.array(
    [[record["annotations"][0]["K"][0],         0,
record["annotations"][0]["K"][1]],
     [        0, record["annotations"][0]["K"][0],
record["annotations"][0]["K"][2]],
     [        0,         0,      1]],
    dtype=np.float32
)

# Load the obj file.
v, f = load_obj(obj_file_name)

# Read the image.
img = imread(image_path)

# Convert the vertices to camera coordinates.
v = convert_vert_to_camera_coords(v, K, R, t)

# Plot the contour on the image.
img = visualize_contour(img, v, f, "obj")

# Visualize the image.
cv2_imshow(img)

def convert_vert_to_camera_coords(v, K, R, t):
    """Convert the vertices in world coordinate to camera coordinates.

    Args:
        v: The vertices array.
        K: The camera matrix.
        R: The rotation matrix. (3 X 3 list)
        t: The translation matrix. (3 list)

    Outputs:
        v: The camera coordinate.
    """

    # Make the vertice upside down.

    # Convert R and t to numpy array.
    R = np.asarray(R)
    t = np.asarray(t)

    # Compute the mesh vertices positions on the image.
    Rt = np.eye(4)
    Rt[:3, 3] = t
    Rt[:3, :3] = R
    Rt = Rt[:3, :] # Convert 4 X 4 to 3 X 4
    P = np.ones((v.shape[0], v.shape[1]+1))
    P[:, :-1] = v
    P = P.T # 4 X N
    img_cor_points = np.dot(K, np.dot(Rt, P))
    img_cor_points = img_cor_points.T # N X 3
    img_cor_points[:, 0] /= img_cor_points[:, 2]
    img_cor_points[:, 1] /= img_cor_points[:, 2]

    # Convert the coordinates to pixel level.
    img_cor_points = img_cor_points.astype(int)

    return img_cor_points

Add depth channel

Hi,

I was trying to add one more channel at the input. That channel is a depth channel. I hope by adding it the mesh quality could be improved.

But it turns out that the results of adding one more channel are making the results worse. The box AP, mask AP, and mesh AP are worse.

My question is:

1. Is it because of the loaded ResNet?
2. During training, I am not sure whether the weights of ResNet is open or frozen. Could you help me with it? I saw some parameters called DEFORM in the yaml file.

Best regards

Nan values in the loss dict

FloatingPointError: Loss became infinite or NaN at iteration=8! loss_dict = {'loss_cls': tensor(0.9525, device='cuda:1', grad_fn=<NllLossBackward>), 'loss_box_reg': tensor(0., device='cuda:1', grad_fn=<DivBackward0>), 'loss_z_reg': tensor(nan, device='cuda:1', grad_fn=<DivBackward0>), 'loss_mask': tensor(5.8197, device='cuda:1', grad_fn=<BinaryCrossEntropyWithLogitsBackward>), 'loss_voxel': tensor(2.0721, device='cuda:1', grad_fn=<MulBackward0>), 'loss_chamfer': tensor(0., device='cuda:1', grad_fn=<MulBackward0>), 'loss_normals': tensor(0., device='cuda:1', grad_fn=<MulBackward0>), 'loss_edge': tensor(0., device='cuda:1', grad_fn=<MulBackward0>), 'loss_rpn_cls': tensor(0.3055, device='cuda:1', grad_fn=<MulBackward0>), 'loss_rpn_loc': tensor(0.0218, device='cuda:1', grad_fn=<MulBackward0>)}

I tried reducing the BASE_LR in config.yaml(as suggested in #36) but still face similar error. Can you suggest something I can try?

Optimal 3D object file format for meshrcnn

Hello,

I hope this is the proper forum for this question, if not I am apologize in advance.

I am creating a 3D object reverse search engine and have been looking at Mesh R-CNN for this with PyTorch model tracing and potentially FAISS.

My question is, are there particular 3D file storage formats that are superior to others? e.g. I have a 3D scanner that can output various filetypes including OBJ and PLY; is there a benefit to one over the other?

I am correct in assuming that I could easily train Mesh R-CNN on my dataset of 3D objects as long as the models are clean with fixed mesh and stored in either PLY or OBJ format?

Thank you in advance

paulo

Question about datasets of pix3D

I run the command line in "INSTRUCTIONS_PIX3D.md" to train the model.

There was something wrong during the training process:
"SyntaxError: not a TIFF file (header b'' not valid)."

I checked up the datasets and found that "exif" message of some images caused this error. I deleted these images‘ "exif" message. Then training process worked.

However, I was wondering why I met this question, and deleting information of some data is a right method or not.
Is there something wrong in pix3d's datasets? Or it is just because my environment is Incorrect?

How to train new categories

Hi, thank you for sharing your work.It seems useful.
Or how could I train this network for new(other) categories?

GPU utilization, latency during inference and model accuracy

Hello,
I trained a model on my 2-GPU machine, basically to reproduce the results of the meshrcnn_R50 that's shared here, without modifying anything.

python tools/train_net.py --num-gpus 2 --config-file configs/pix3d/meshrcnn_R50_FPN.yaml

I ran the evaluation both with the shared meshrcnn_R50.pth and the model I trained, and it seems that the accuracy of the model is 10% worse using my model. What's stranger is that the GPU utilization and the time for inference is 4x worse. Below, I have shared the stats for inference on 8 images from Pix3D.

time python tools/train_net.py --num-gpus 2 --config-file configs/pix3d/meshrcnn_R50_FPN.yaml --eval-only MODEL.WEIGHTS output/meshrcnn_R50.pth

real 0m15.656s
user 0m35.867s
sys 0m7.237s
[02/28 18:22:14 meshrcnn.evaluation.pix3d_evaluation]: Box AP 0.92593
[02/28 18:22:14 meshrcnn.evaluation.pix3d_evaluation]: Mask AP 0.92593
[02/28 18:22:14 meshrcnn.evaluation.pix3d_evaluation]: Mesh AP 0.44444
GPU util: 2.8-2.9GB per machine, as per nvidia-smi

time python tools/train_net.py --num-gpus 2 --config-file configs/pix3d/meshrcnn_R50_FPN.yaml --eval-only MODEL.WEIGHTS output/model_final.pth

real 0m35.237s
user 1m5.214s
sys 0m9.366s
[02/28 18:20:49 meshrcnn.evaluation.pix3d_evaluation]: Box AP 0.86772
[02/28 18:20:49 meshrcnn.evaluation.pix3d_evaluation]: Mask AP 0.75661
[02/28 18:20:49 meshrcnn.evaluation.pix3d_evaluation]: Mesh AP 0.40432
GPU util: 8GB per machine, as per nvidia-smi

Please let me know if you have any ideas of why this might happen. I have also shared the detectron2 collect_env output below

---

sys.platform linux
Python 3.7.6 (default, Jan 8 2020, 19:59:22) [GCC 7.3.0]
numpy 1.17.4
detectron2 0.1 @/home/USER/detectron2/detectron2
detectron2 compiler GCC 5.5
detectron2 CUDA compiler 10.1
detectron2 arch flags sm_50, sm_52, sm_60, sm_61, sm_70, sm_75
DETECTRON2_ENV_MODULE
PyTorch 1.4.0 @/home/USER/.conda/envs/meshrcnn1/lib/python3.7/site-packages/torch
PyTorch debug build False
CUDA available True
GPU 0 TITAN X (Pascal)
GPU 1 GeForce GTX TITAN X
CUDA_HOME /usr/local/cuda
NVCC Cuda compilation tools, release 10.1, V10.1.243
Pillow 7.0.0
torchvision 0.5.0 @/home/USER/.conda/envs/meshrcnn1/lib/python3.7/site-packages/torchvision
torchvision arch flags sm_35, sm_50, sm_60, sm_70, sm_75
cv2 4.2.0

---

PyTorch built with:

• GCC 7.3
• Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
• Intel(R) MKL-DNN v0.21.1 (Git Hash 7d2fd500bc78936d1d648ca713b901012f470dbc)
• OpenMP 201511 (a.k.a. OpenMP 4.5)
• NNPACK is enabled
• CUDA Runtime 10.1
• NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_37,code=compute_37
• CuDNN 7.6.3
• Magma 2.5.1
• Build settings: BLAS=MKL, BUILD_NAMEDTENSOR=OFF, BUILD_TYPE=Release, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -fopenmp -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Wno-stringop-overflow, DISABLE_NUMA=1, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, USE_CUDA=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_STATIC_DISPATCH=OFF,

FloatingPointError: Loss became infinite or NaN at iteration=1002!

[05/27 09:32:48 detectron2]: Rank of current process: 0. World size: 1
[05/27 09:32:48 detectron2]: Environment info:

---

sys.platform linux
Python 3.7.7 (default, Mar 26 2020, 15:48:22) [GCC 7.3.0]
numpy 1.18.1
detectron2 0.1.2 @./detectron2-master/detectron2
detectron2 compiler GCC 7.5
detectron2 CUDA compiler 10.0
detectron2 arch flags sm_75
DETECTRON2_ENV_MODULE
PyTorch 1.4.0 @python3.7/site-packages/torch
PyTorch debug build False
CUDA available True
GPU 0 Graphics Device
CUDA_HOME /usr/local/cuda
NVCC Cuda compilation tools, release 10.0, V10.0.130
Pillow 7.1.2
torchvision 0.5.0 @python3.7/site-packages/torchvision
torchvision arch flags sm_35, sm_50, sm_60, sm_70, sm_75
fvcore 0.1.1.post200513
cv2 4.2.0

---

PyTorch built with:

• GCC 7.3
• Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
• Intel(R) MKL-DNN v0.21.1 (Git Hash 7d2fd500bc78936d1d648ca713b901012f470dbc)
• OpenMP 201511 (a.k.a. OpenMP 4.5)
• NNPACK is enabled
• CUDA Runtime 10.0
• NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_37,code=compute_37
• CuDNN 7.6.3
• Magma 2.5.1
• Build settings: BLAS=MKL, BUILD_NAMEDTENSOR=OFF, BUILD_TYPE=Release, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -fopenmp -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Wno-stringop-overflow, DISABLE_NUMA=1, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, USE_CUDA=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_STATIC_DISPATCH=OFF,

[05/27 09:32:48 detectron2]: Command line arguments: Namespace(config_file='/meshrcnn/configs/pix3d/meshrcnn_R50_FPN_HD.yaml', dist_url='tcp://127.0.0.1:10112', eval_only=False, machine_rank=0, num_gpus=1, num_machines=1, opts=[], resume=False)
[05/27 09:32:48 detectron2]: Contents of args.config_file=/meshrcnn/configs/pix3d/meshrcnn_R50_FPN_HD.yaml:
BASE: "MESH-RCNN-FPN_HD.yaml"
MODEL:
WEIGHTS: "/meshrcnn/R-50.pkl"
MASK_ON: True
VOXEL_ON: True
MESH_ON: True
ZPRED_ON: True
RESNETS:
DEPTH: 50
RPN:
IOU_THRESHOLDS: [0.2, 0.5, 0.7]
IOU_LABELS: [-1, 0, -1, 1]
SMOOTH_L1_BETA: 0.111
ROI_HEADS:
NAME: "MeshRCNNROIHeads"
BATCH_SIZE_PER_IMAGE: 64
NUM_CLASSES: 2 #9 # Number of foreground classes
IOU_THRESHOLDS: [0.2, 0.5]
IOU_LABELS: [-1, 0, 1]
ROI_BOX_HEAD:
SMOOTH_L1_BETA: 1.0
ROI_Z_HEAD:
NAME: "FastRCNNFCHead"
Z_REG_WEIGHT: 1.0
SMOOTH_L1_BETA: 1.0
ROI_MASK_HEAD:
NAME: "MaskRCNNConvUpsampleHead"
POOLER_RESOLUTION: 14
POOLER_SAMPLING_RATIO: 2
NUM_CONV: 4
ROI_VOXEL_HEAD:
NAME: "VoxelRCNNConvUpsampleHead"
POOLER_RESOLUTION: 12
POOLER_SAMPLING_RATIO: 2
NUM_CONV: 4
NUM_DEPTH: 24
CLS_AGNOSTIC_VOXEL: True
LOSS_WEIGHT: 3.0
CUBIFY_THRESH: 0.2
ROI_MESH_HEAD:
NAME: "MeshRCNNGraphConvHead"
POOLER_RESOLUTION: 14
POOLER_SAMPLING_RATIO: 2
NUM_STAGES: 3
NUM_GRAPH_CONVS: 3
GRAPH_CONV_DIM: 128
GRAPH_CONV_INIT: "normal"
GT_COORD_THRESH: 5.0
CHAMFER_LOSS_WEIGHT: 1.0
NORMALS_LOSS_WEIGHT: 0.1
EDGE_LOSS_WEIGHT: 1.0
DATASETS:
TRAIN: ("bed_bookcase_s1_train",)
TEST: ("bed_bookcase_s1_test",)
SOLVER:
BASE_LR: 0.0025 #0.02
WEIGHT_DECAY: 0.0001
STEPS: (64000, 80000) #(8000, 10000)
MAX_ITER: 88000 # 11000
WARMUP_ITERS: 1000
WARMUP_FACTOR: 0.1
IMS_PER_BATCH: 1

[05/27 09:32:48 detectron2]: Running with full config:
CUDNN_BENCHMARK: False
DATALOADER:
ASPECT_RATIO_GROUPING: False
FILTER_EMPTY_ANNOTATIONS: True
NUM_WORKERS: 4
REPEAT_THRESHOLD: 0.0
SAMPLER_TRAIN: TrainingSampler
DATASETS:
PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000
PRECOMPUTED_PROPOSAL_TOPK_TRAIN: 2000
PROPOSAL_FILES_TEST: ()
PROPOSAL_FILES_TRAIN: ()
TEST: ('bed_bookcase_s1_test',)
TRAIN: ('bed_bookcase_s1_train',)
GLOBAL:
HACK: 1.0
INPUT:
CROP:
ENABLED: False
SIZE: [0.9, 0.9]
TYPE: relative_range
FORMAT: BGR
MASK_FORMAT: polygon
MAX_SIZE_TEST: 1333
MAX_SIZE_TRAIN: 1333
MIN_SIZE_TEST: 800
MIN_SIZE_TRAIN: (800,)
MIN_SIZE_TRAIN_SAMPLING: choice
MODEL:
ANCHOR_GENERATOR:
ANGLES: [[-90, 0, 90]]
ASPECT_RATIOS: [[0.5, 1.0, 2.0]]
NAME: DefaultAnchorGenerator
OFFSET: 0.0
SIZES: [[32], [64], [128], [256], [512]]
BACKBONE:
FREEZE_AT: 2
NAME: build_resnet_fpn_backbone
DEVICE: cuda
FPN:
FUSE_TYPE: sum
IN_FEATURES: ['res2', 'res3', 'res4', 'res5']
NORM:
OUT_CHANNELS: 256
KEYPOINT_ON: False
LOAD_PROPOSALS: False
MASK_ON: True
MESH_ON: True
META_ARCHITECTURE: GeneralizedRCNN
PANOPTIC_FPN:
COMBINE:
ENABLED: True
INSTANCES_CONFIDENCE_THRESH: 0.5
OVERLAP_THRESH: 0.5
STUFF_AREA_LIMIT: 4096
INSTANCE_LOSS_WEIGHT: 1.0
PIXEL_MEAN: [103.53, 116.28, 123.675]
PIXEL_STD: [1.0, 1.0, 1.0]
PROPOSAL_GENERATOR:
MIN_SIZE: 0
NAME: RPN
RESNETS:
DEFORM_MODULATED: False
DEFORM_NUM_GROUPS: 1
DEFORM_ON_PER_STAGE: [False, False, False, False]
DEPTH: 50
NORM: FrozenBN
NUM_GROUPS: 1
OUT_FEATURES: ['res2', 'res3', 'res4', 'res5']
RES2_OUT_CHANNELS: 256
RES5_DILATION: 1
STEM_OUT_CHANNELS: 64
STRIDE_IN_1X1: True
WIDTH_PER_GROUP: 64
RETINANET:
BBOX_REG_WEIGHTS: (1.0, 1.0, 1.0, 1.0)
FOCAL_LOSS_ALPHA: 0.25
FOCAL_LOSS_GAMMA: 2.0
IN_FEATURES: ['p3', 'p4', 'p5', 'p6', 'p7']
IOU_LABELS: [0, -1, 1]
IOU_THRESHOLDS: [0.4, 0.5]
NMS_THRESH_TEST: 0.5
NUM_CLASSES: 80
NUM_CONVS: 4
PRIOR_PROB: 0.01
SCORE_THRESH_TEST: 0.05
SMOOTH_L1_LOSS_BETA: 0.1
TOPK_CANDIDATES_TEST: 1000
ROI_BOX_CASCADE_HEAD:
BBOX_REG_WEIGHTS: ((10.0, 10.0, 5.0, 5.0), (20.0, 20.0, 10.0, 10.0), (30.0, 30.0, 15.0, 15.0))
IOUS: (0.5, 0.6, 0.7)
ROI_BOX_HEAD:
BBOX_REG_WEIGHTS: (10.0, 10.0, 5.0, 5.0)
CLS_AGNOSTIC_BBOX_REG: False
CONV_DIM: 256
FC_DIM: 1024
NAME: FastRCNNConvFCHead
NORM:
NUM_CONV: 0
NUM_FC: 2
POOLER_RESOLUTION: 7
POOLER_SAMPLING_RATIO: 2
POOLER_TYPE: ROIAlign
SMOOTH_L1_BETA: 1.0
TRAIN_ON_PRED_BOXES: False
ROI_HEADS:
BATCH_SIZE_PER_IMAGE: 64
IN_FEATURES: ['p2', 'p3', 'p4', 'p5']
IOU_LABELS: [-1, 0, 1]
IOU_THRESHOLDS: [0.2, 0.5]
NAME: MeshRCNNROIHeads
NMS_THRESH_TEST: 0.5
NUM_CLASSES: 2
POSITIVE_FRACTION: 0.25
PROPOSAL_APPEND_GT: True
SCORE_THRESH_TEST: 0.05
ROI_KEYPOINT_HEAD:
CONV_DIMS: (512, 512, 512, 512, 512, 512, 512, 512)
LOSS_WEIGHT: 1.0
MIN_KEYPOINTS_PER_IMAGE: 1
NAME: KRCNNConvDeconvUpsampleHead
NORMALIZE_LOSS_BY_VISIBLE_KEYPOINTS: True
NUM_KEYPOINTS: 17
POOLER_RESOLUTION: 14
POOLER_SAMPLING_RATIO: 0
POOLER_TYPE: ROIAlignV2
ROI_MASK_HEAD:
CLS_AGNOSTIC_MASK: False
CONV_DIM: 256
NAME: MaskRCNNConvUpsampleHead
NORM:
NUM_CONV: 4
POOLER_RESOLUTION: 14
POOLER_SAMPLING_RATIO: 2
POOLER_TYPE: ROIAlign
ROI_MESH_HEAD:
CHAMFER_LOSS_WEIGHT: 1.0
EDGE_LOSS_WEIGHT: 1.0
GRAPH_CONV_DIM: 128
GRAPH_CONV_INIT: normal
GT_COORD_THRESH: 5.0
GT_NUM_SAMPLES: 5000
ICO_SPHERE_LEVEL: -1
NAME: MeshRCNNGraphConvHead
NORMALS_LOSS_WEIGHT: 0.1
NUM_GRAPH_CONVS: 3
NUM_STAGES: 3
POOLER_RESOLUTION: 14
POOLER_SAMPLING_RATIO: 2
POOLER_TYPE: ROIAlign
PRED_NUM_SAMPLES: 5000
ROI_VOXEL_HEAD:
CLS_AGNOSTIC_VOXEL: True
CONV_DIM: 256
CUBIFY_THRESH: 0.2
LOSS_WEIGHT: 3.0
NAME: VoxelRCNNConvUpsampleHead
NORM:
NUM_CONV: 4
NUM_DEPTH: 24
POOLER_RESOLUTION: 12
POOLER_SAMPLING_RATIO: 2
POOLER_TYPE: ROIAlign
ROI_Z_HEAD:
CLS_AGNOSTIC_Z_REG: False
FC_DIM: 1024
NAME: FastRCNNFCHead
NUM_FC: 2
POOLER_RESOLUTION: 7
POOLER_SAMPLING_RATIO: 2
POOLER_TYPE: ROIAlign
SMOOTH_L1_BETA: 1.0
Z_REG_WEIGHT: 1.0
RPN:
BATCH_SIZE_PER_IMAGE: 256
BBOX_REG_WEIGHTS: (1.0, 1.0, 1.0, 1.0)
BOUNDARY_THRESH: -1
HEAD_NAME: StandardRPNHead
IN_FEATURES: ['p2', 'p3', 'p4', 'p5', 'p6']
IOU_LABELS: [-1, 0, -1, 1]
IOU_THRESHOLDS: [0.2, 0.5, 0.7]
LOSS_WEIGHT: 1.0
NMS_THRESH: 0.7
POSITIVE_FRACTION: 0.5
POST_NMS_TOPK_TEST: 1000
POST_NMS_TOPK_TRAIN: 1000
PRE_NMS_TOPK_TEST: 1000
PRE_NMS_TOPK_TRAIN: 2000
SMOOTH_L1_BETA: 0.111
SEM_SEG_HEAD:
COMMON_STRIDE: 4
CONVS_DIM: 128
IGNORE_VALUE: 255
IN_FEATURES: ['p2', 'p3', 'p4', 'p5']
LOSS_WEIGHT: 1.0
NAME: SemSegFPNHead
NORM: GN
NUM_CLASSES: 54
VIS_MINIBATCH: False
VOXEL_ON: True
WEIGHTS: /meshrcnn/R-50.pkl
ZPRED_ON: True
OUTPUT_DIR: ./output
SEED: -1
SOLVER:
BASE_LR: 0.0025
BIAS_LR_FACTOR: 1.0
CHECKPOINT_PERIOD: 5000
CLIP_GRADIENTS:
CLIP_TYPE: value
CLIP_VALUE: 1.0
ENABLED: False
NORM_TYPE: 2.0
GAMMA: 0.1
IMS_PER_BATCH: 1
LR_SCHEDULER_NAME: WarmupMultiStepLR
MAX_ITER: 88000
MOMENTUM: 0.9
NESTEROV: False
STEPS: (64000, 80000)
WARMUP_FACTOR: 0.1
WARMUP_ITERS: 1000
WARMUP_METHOD: linear
WEIGHT_DECAY: 0.0001
WEIGHT_DECAY_BIAS: 0.0001
WEIGHT_DECAY_NORM: 0.0
TEST:
AUG:
ENABLED: False
FLIP: True
MAX_SIZE: 4000
MIN_SIZES: (400, 500, 600, 700, 800, 900, 1000, 1100, 1200)
DETECTIONS_PER_IMAGE: 100
EVAL_PERIOD: 0
EXPECTED_RESULTS: []
KEYPOINT_OKS_SIGMAS: []
PRECISE_BN:
ENABLED: False
NUM_ITER: 200
VERSION: 2
VIS_PERIOD: 0
[05/27 09:32:48 detectron2]: Full config saved to ./output/config.yaml
[05/27 09:32:48 d2.utils.env]: Using a generated random seed 48937753
[05/27 09:32:51 d2.engine.defaults]: Model:
GeneralizedRCNN(
(backbone): FPN(
(fpn_lateral2): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1))
(fpn_output2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(fpn_lateral3): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1))
(fpn_output3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(fpn_lateral4): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1))
(fpn_output4): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(fpn_lateral5): Conv2d(2048, 256, kernel_size=(1, 1), stride=(1, 1))
(fpn_output5): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(top_block): LastLevelMaxPool()
(bottom_up): ResNet(
(stem): BasicStem(
(conv1): Conv2d(
3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False
(norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
)
)
(res2): Sequential(
(0): BottleneckBlock(
(shortcut): Conv2d(
64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
)
(conv1): Conv2d(
64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
)
(conv2): Conv2d(
64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
)
(conv3): Conv2d(
64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
)
)
(1): BottleneckBlock(
(conv1): Conv2d(
256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
)
(conv2): Conv2d(
64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
)
(conv3): Conv2d(
64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
)
)
(2): BottleneckBlock(
(conv1): Conv2d(
256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
)
(conv2): Conv2d(
64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
)
(conv3): Conv2d(
64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
)
)
)
(res3): Sequential(
(0): BottleneckBlock(
(shortcut): Conv2d(
256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False
(norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
)
(conv1): Conv2d(
256, 128, kernel_size=(1, 1), stride=(2, 2), bias=False
(norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
)
(conv2): Conv2d(
128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
)
(conv3): Conv2d(
128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
)
)
(1): BottleneckBlock(
(conv1): Conv2d(
512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
)
(conv2): Conv2d(
128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
)
(conv3): Conv2d(
128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
)
)
(2): BottleneckBlock(
(conv1): Conv2d(
512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
)
(conv2): Conv2d(
128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
)
(conv3): Conv2d(
128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
)
)
(3): BottleneckBlock(
(conv1): Conv2d(
512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
)
(conv2): Conv2d(
128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
)
(conv3): Conv2d(
128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
)
)
)
(res4): Sequential(
(0): BottleneckBlock(
(shortcut): Conv2d(
512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False
(norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
)
(conv1): Conv2d(
512, 256, kernel_size=(1, 1), stride=(2, 2), bias=False
(norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
)
(conv2): Conv2d(
256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
)
(conv3): Conv2d(
256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
)
)
(1): BottleneckBlock(
(conv1): Conv2d(
1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
)
(conv2): Conv2d(
256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
)
(conv3): Conv2d(
256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
)
)
(2): BottleneckBlock(
(conv1): Conv2d(
1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
)
(conv2): Conv2d(
256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
)
(conv3): Conv2d(
256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
)
)
(3): BottleneckBlock(
(conv1): Conv2d(
1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
)
(conv2): Conv2d(
256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
)
(conv3): Conv2d(
256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
)
)
(4): BottleneckBlock(
(conv1): Conv2d(
1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
)
(conv2): Conv2d(
256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
)
(conv3): Conv2d(
256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
)
)
(5): BottleneckBlock(
(conv1): Conv2d(
1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
)
(conv2): Conv2d(
256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
)
(conv3): Conv2d(
256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
)
)
)
(res5): Sequential(
(0): BottleneckBlock(
(shortcut): Conv2d(
1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False
(norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
)
(conv1): Conv2d(
1024, 512, kernel_size=(1, 1), stride=(2, 2), bias=False
(norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
)
(conv2): Conv2d(
512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
)
(conv3): Conv2d(
512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
)
)
(1): BottleneckBlock(
(conv1): Conv2d(
2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
)
(conv2): Conv2d(
512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
)
(conv3): Conv2d(
512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
)
)
(2): BottleneckBlock(
(conv1): Conv2d(
2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
)
(conv2): Conv2d(
512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
)
(conv3): Conv2d(
512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
)
)
)
)
)
(proposal_generator): RPN(
(anchor_generator): DefaultAnchorGenerator(
(cell_anchors): BufferList()
)
(rpn_head): StandardRPNHead(
(conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(objectness_logits): Conv2d(256, 3, kernel_size=(1, 1), stride=(1, 1))
(anchor_deltas): Conv2d(256, 12, kernel_size=(1, 1), stride=(1, 1))
)
)
(roi_heads): MeshRCNNROIHeads(
(box_pooler): ROIPooler(
(level_poolers): ModuleList(
(0): ROIAlign(output_size=(7, 7), spatial_scale=0.25, sampling_ratio=2, aligned=False)
(1): ROIAlign(output_size=(7, 7), spatial_scale=0.125, sampling_ratio=2, aligned=False)
(2): ROIAlign(output_size=(7, 7), spatial_scale=0.0625, sampling_ratio=2, aligned=False)
(3): ROIAlign(output_size=(7, 7), spatial_scale=0.03125, sampling_ratio=2, aligned=False)
)
)
(box_head): FastRCNNConvFCHead(
(fc1): Linear(in_features=12544, out_features=1024, bias=True)
(fc2): Linear(in_features=1024, out_features=1024, bias=True)
)
(box_predictor): FastRCNNOutputLayers(
(cls_score): Linear(in_features=1024, out_features=3, bias=True)
(bbox_pred): Linear(in_features=1024, out_features=8, bias=True)
)
(mask_pooler): ROIPooler(
(level_poolers): ModuleList(
(0): ROIAlign(output_size=(14, 14), spatial_scale=0.25, sampling_ratio=2, aligned=False)
(1): ROIAlign(output_size=(14, 14), spatial_scale=0.125, sampling_ratio=2, aligned=False)
(2): ROIAlign(output_size=(14, 14), spatial_scale=0.0625, sampling_ratio=2, aligned=False)
(3): ROIAlign(output_size=(14, 14), spatial_scale=0.03125, sampling_ratio=2, aligned=False)
)
)
(mask_head): MaskRCNNConvUpsampleHead(
(mask_fcn1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(mask_fcn2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(mask_fcn3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(mask_fcn4): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(deconv): ConvTranspose2d(256, 256, kernel_size=(2, 2), stride=(2, 2))
(predictor): Conv2d(256, 2, kernel_size=(1, 1), stride=(1, 1))
)
(z_pooler): ROIPooler(
(level_poolers): ModuleList(
(0): ROIAlign(output_size=(7, 7), spatial_scale=0.25, sampling_ratio=2, aligned=False)
(1): ROIAlign(output_size=(7, 7), spatial_scale=0.125, sampling_ratio=2, aligned=False)
(2): ROIAlign(output_size=(7, 7), spatial_scale=0.0625, sampling_ratio=2, aligned=False)
(3): ROIAlign(output_size=(7, 7), spatial_scale=0.03125, sampling_ratio=2, aligned=False)
)
)
(z_head): FastRCNNFCHead(
(z_fc1): Linear(in_features=12544, out_features=1024, bias=True)
(z_fc2): Linear(in_features=1024, out_features=1024, bias=True)
(z_pred): Linear(in_features=1024, out_features=2, bias=True)
)
(voxel_pooler): ROIPooler(
(level_poolers): ModuleList(
(0): ROIAlign(output_size=(12, 12), spatial_scale=0.25, sampling_ratio=2, aligned=False)
(1): ROIAlign(output_size=(12, 12), spatial_scale=0.125, sampling_ratio=2, aligned=False)
(2): ROIAlign(output_size=(12, 12), spatial_scale=0.0625, sampling_ratio=2, aligned=False)
(3): ROIAlign(output_size=(12, 12), spatial_scale=0.03125, sampling_ratio=2, aligned=False)
)
)
(voxel_head): VoxelRCNNConvUpsampleHead(
(voxel_fcn1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(voxel_fcn2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(voxel_fcn3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(voxel_fcn4): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(deconv): ConvTranspose2d(256, 256, kernel_size=(2, 2), stride=(2, 2))
(predictor): Conv2d(256, 24, kernel_size=(1, 1), stride=(1, 1))
)
(mesh_pooler): ROIPooler(
(level_poolers): ModuleList(
(0): ROIAlign(output_size=(14, 14), spatial_scale=0.25, sampling_ratio=2, aligned=False)
(1): ROIAlign(output_size=(14, 14), spatial_scale=0.125, sampling_ratio=2, aligned=False)
(2): ROIAlign(output_size=(14, 14), spatial_scale=0.0625, sampling_ratio=2, aligned=False)
(3): ROIAlign(output_size=(14, 14), spatial_scale=0.03125, sampling_ratio=2, aligned=False)
)
)
(mesh_head): MeshRCNNGraphConvHead(
(stages): ModuleList(
(0): MeshRefinementStage(
(bottleneck): Linear(in_features=256, out_features=128, bias=True)
(verts_offset): Linear(in_features=131, out_features=3, bias=True)
(gconvs): ModuleList(
(0): GraphConv(131 -> 128, directed=False)
(1): GraphConv(131 -> 128, directed=False)
(2): GraphConv(131 -> 128, directed=False)
)
)
(1): MeshRefinementStage(
(bottleneck): Linear(in_features=256, out_features=128, bias=True)
(verts_offset): Linear(in_features=131, out_features=3, bias=True)
(gconvs): ModuleList(
(0): GraphConv(259 -> 128, directed=False)
(1): GraphConv(131 -> 128, directed=False)
(2): GraphConv(131 -> 128, directed=False)
)
)
(2): MeshRefinementStage(
(bottleneck): Linear(in_features=256, out_features=128, bias=True)
(verts_offset): Linear(in_features=131, out_features=3, bias=True)
(gconvs): ModuleList(
(0): GraphConv(259 -> 128, directed=False)
(1): GraphConv(131 -> 128, directed=False)
(2): GraphConv(131 -> 128, directed=False)
)
)
)
)
)
)
[05/27 09:32:51 d2.data.detection_utils]: TransformGens used in training: [ResizeShortestEdge(short_edge_length=(800,), max_size=1333, sample_style='choice'), RandomFlip()]
bed_bookcase_s1_train
[05/27 09:32:51 meshrcnn.data.meshrcnn_transforms]: Loading models from bed_bookcase_s1_train...
/home/hhd/Documents/paper/pytorch3d-master/pytorch3d/io/obj_io.py:70: UserWarning: Faces have invalid indices
warnings.warn("Faces have invalid indices")
[05/27 09:32:53 meshrcnn.data.meshrcnn_transforms]: Unique objects loaded: 34
loading annotations into memory...
Done (t=0.01s)
creating index...
index created!
thing_classes: ['bed', 'bookcase']
[05/27 09:32:53 meshrcnn.data.datasets.pix3d]: Loaded 1063 images in COCO format from /meshrcnn/datasets/pix3d_hd/bed_bookcase_s1_train.json
[05/27 09:32:53 d2.data.build]: Removed 0 images with no usable annotations. 1063 images left.
[05/27 09:32:53 d2.data.build]: Distribution of instances among all 2 categories:

category	#instances	category	#instances
bed	781	bookcase	282
total	1063
[05/27 09:32:53 d2.data.common]: Serializing 1063 elements to byte tensors and concatenating them all ...
[05/27 09:32:53 d2.data.common]: Serialized dataset takes 0.68 MiB
[05/27 09:32:53 d2.data.build]: Using training sampler TrainingSampler
[05/27 09:32:54 fvcore.common.checkpoint]: Loading checkpoint from /meshrcnn/R-50.pkl
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: Remapping C2 weights ......
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.0.conv1.norm.bias loaded from res2_0_branch2a_bn_beta of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.0.conv1.norm.running_mean loaded from res2_0_branch2a_bn_running_mean of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.0.conv1.norm.running_var loaded from res2_0_branch2a_bn_running_var of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.0.conv1.norm.weight loaded from res2_0_branch2a_bn_gamma of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.0.conv1.weight loaded from res2_0_branch2a_w of shape (64, 64, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.0.conv2.norm.bias loaded from res2_0_branch2b_bn_beta of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.0.conv2.norm.running_mean loaded from res2_0_branch2b_bn_running_mean of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.0.conv2.norm.running_var loaded from res2_0_branch2b_bn_running_var of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.0.conv2.norm.weight loaded from res2_0_branch2b_bn_gamma of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.0.conv2.weight loaded from res2_0_branch2b_w of shape (64, 64, 3, 3)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.0.conv3.norm.bias loaded from res2_0_branch2c_bn_beta of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.0.conv3.norm.running_mean loaded from res2_0_branch2c_bn_running_mean of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.0.conv3.norm.running_var loaded from res2_0_branch2c_bn_running_var of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.0.conv3.norm.weight loaded from res2_0_branch2c_bn_gamma of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.0.conv3.weight loaded from res2_0_branch2c_w of shape (256, 64, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.0.shortcut.norm.bias loaded from res2_0_branch1_bn_beta of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.0.shortcut.norm.running_mean loaded from res2_0_branch1_bn_running_mean of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.0.shortcut.norm.running_var loaded from res2_0_branch1_bn_running_var of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.0.shortcut.norm.weight loaded from res2_0_branch1_bn_gamma of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.0.shortcut.weight loaded from res2_0_branch1_w of shape (256, 64, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.1.conv1.norm.bias loaded from res2_1_branch2a_bn_beta of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.1.conv1.norm.running_mean loaded from res2_1_branch2a_bn_running_mean of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.1.conv1.norm.running_var loaded from res2_1_branch2a_bn_running_var of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.1.conv1.norm.weight loaded from res2_1_branch2a_bn_gamma of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.1.conv1.weight loaded from res2_1_branch2a_w of shape (64, 256, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.1.conv2.norm.bias loaded from res2_1_branch2b_bn_beta of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.1.conv2.norm.running_mean loaded from res2_1_branch2b_bn_running_mean of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.1.conv2.norm.running_var loaded from res2_1_branch2b_bn_running_var of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.1.conv2.norm.weight loaded from res2_1_branch2b_bn_gamma of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.1.conv2.weight loaded from res2_1_branch2b_w of shape (64, 64, 3, 3)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.1.conv3.norm.bias loaded from res2_1_branch2c_bn_beta of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.1.conv3.norm.running_mean loaded from res2_1_branch2c_bn_running_mean of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.1.conv3.norm.running_var loaded from res2_1_branch2c_bn_running_var of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.1.conv3.norm.weight loaded from res2_1_branch2c_bn_gamma of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.1.conv3.weight loaded from res2_1_branch2c_w of shape (256, 64, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.2.conv1.norm.bias loaded from res2_2_branch2a_bn_beta of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.2.conv1.norm.running_mean loaded from res2_2_branch2a_bn_running_mean of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.2.conv1.norm.running_var loaded from res2_2_branch2a_bn_running_var of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.2.conv1.norm.weight loaded from res2_2_branch2a_bn_gamma of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.2.conv1.weight loaded from res2_2_branch2a_w of shape (64, 256, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.2.conv2.norm.bias loaded from res2_2_branch2b_bn_beta of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.2.conv2.norm.running_mean loaded from res2_2_branch2b_bn_running_mean of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.2.conv2.norm.running_var loaded from res2_2_branch2b_bn_running_var of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.2.conv2.norm.weight loaded from res2_2_branch2b_bn_gamma of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.2.conv2.weight loaded from res2_2_branch2b_w of shape (64, 64, 3, 3)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.2.conv3.norm.bias loaded from res2_2_branch2c_bn_beta of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.2.conv3.norm.running_mean loaded from res2_2_branch2c_bn_running_mean of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.2.conv3.norm.running_var loaded from res2_2_branch2c_bn_running_var of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.2.conv3.norm.weight loaded from res2_2_branch2c_bn_gamma of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res2.2.conv3.weight loaded from res2_2_branch2c_w of shape (256, 64, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.0.conv1.norm.bias loaded from res3_0_branch2a_bn_beta of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.0.conv1.norm.running_mean loaded from res3_0_branch2a_bn_running_mean of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.0.conv1.norm.running_var loaded from res3_0_branch2a_bn_running_var of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.0.conv1.norm.weight loaded from res3_0_branch2a_bn_gamma of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.0.conv1.weight loaded from res3_0_branch2a_w of shape (128, 256, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.0.conv2.norm.bias loaded from res3_0_branch2b_bn_beta of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.0.conv2.norm.running_mean loaded from res3_0_branch2b_bn_running_mean of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.0.conv2.norm.running_var loaded from res3_0_branch2b_bn_running_var of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.0.conv2.norm.weight loaded from res3_0_branch2b_bn_gamma of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.0.conv2.weight loaded from res3_0_branch2b_w of shape (128, 128, 3, 3)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.0.conv3.norm.bias loaded from res3_0_branch2c_bn_beta of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.0.conv3.norm.running_mean loaded from res3_0_branch2c_bn_running_mean of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.0.conv3.norm.running_var loaded from res3_0_branch2c_bn_running_var of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.0.conv3.norm.weight loaded from res3_0_branch2c_bn_gamma of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.0.conv3.weight loaded from res3_0_branch2c_w of shape (512, 128, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.0.shortcut.norm.bias loaded from res3_0_branch1_bn_beta of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.0.shortcut.norm.running_mean loaded from res3_0_branch1_bn_running_mean of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.0.shortcut.norm.running_var loaded from res3_0_branch1_bn_running_var of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.0.shortcut.norm.weight loaded from res3_0_branch1_bn_gamma of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.0.shortcut.weight loaded from res3_0_branch1_w of shape (512, 256, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.1.conv1.norm.bias loaded from res3_1_branch2a_bn_beta of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.1.conv1.norm.running_mean loaded from res3_1_branch2a_bn_running_mean of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.1.conv1.norm.running_var loaded from res3_1_branch2a_bn_running_var of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.1.conv1.norm.weight loaded from res3_1_branch2a_bn_gamma of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.1.conv1.weight loaded from res3_1_branch2a_w of shape (128, 512, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.1.conv2.norm.bias loaded from res3_1_branch2b_bn_beta of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.1.conv2.norm.running_mean loaded from res3_1_branch2b_bn_running_mean of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.1.conv2.norm.running_var loaded from res3_1_branch2b_bn_running_var of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.1.conv2.norm.weight loaded from res3_1_branch2b_bn_gamma of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.1.conv2.weight loaded from res3_1_branch2b_w of shape (128, 128, 3, 3)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.1.conv3.norm.bias loaded from res3_1_branch2c_bn_beta of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.1.conv3.norm.running_mean loaded from res3_1_branch2c_bn_running_mean of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.1.conv3.norm.running_var loaded from res3_1_branch2c_bn_running_var of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.1.conv3.norm.weight loaded from res3_1_branch2c_bn_gamma of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.1.conv3.weight loaded from res3_1_branch2c_w of shape (512, 128, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.2.conv1.norm.bias loaded from res3_2_branch2a_bn_beta of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.2.conv1.norm.running_mean loaded from res3_2_branch2a_bn_running_mean of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.2.conv1.norm.running_var loaded from res3_2_branch2a_bn_running_var of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.2.conv1.norm.weight loaded from res3_2_branch2a_bn_gamma of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.2.conv1.weight loaded from res3_2_branch2a_w of shape (128, 512, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.2.conv2.norm.bias loaded from res3_2_branch2b_bn_beta of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.2.conv2.norm.running_mean loaded from res3_2_branch2b_bn_running_mean of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.2.conv2.norm.running_var loaded from res3_2_branch2b_bn_running_var of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.2.conv2.norm.weight loaded from res3_2_branch2b_bn_gamma of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.2.conv2.weight loaded from res3_2_branch2b_w of shape (128, 128, 3, 3)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.2.conv3.norm.bias loaded from res3_2_branch2c_bn_beta of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.2.conv3.norm.running_mean loaded from res3_2_branch2c_bn_running_mean of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.2.conv3.norm.running_var loaded from res3_2_branch2c_bn_running_var of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.2.conv3.norm.weight loaded from res3_2_branch2c_bn_gamma of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.2.conv3.weight loaded from res3_2_branch2c_w of shape (512, 128, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.3.conv1.norm.bias loaded from res3_3_branch2a_bn_beta of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.3.conv1.norm.running_mean loaded from res3_3_branch2a_bn_running_mean of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.3.conv1.norm.running_var loaded from res3_3_branch2a_bn_running_var of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.3.conv1.norm.weight loaded from res3_3_branch2a_bn_gamma of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.3.conv1.weight loaded from res3_3_branch2a_w of shape (128, 512, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.3.conv2.norm.bias loaded from res3_3_branch2b_bn_beta of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.3.conv2.norm.running_mean loaded from res3_3_branch2b_bn_running_mean of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.3.conv2.norm.running_var loaded from res3_3_branch2b_bn_running_var of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.3.conv2.norm.weight loaded from res3_3_branch2b_bn_gamma of shape (128,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.3.conv2.weight loaded from res3_3_branch2b_w of shape (128, 128, 3, 3)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.3.conv3.norm.bias loaded from res3_3_branch2c_bn_beta of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.3.conv3.norm.running_mean loaded from res3_3_branch2c_bn_running_mean of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.3.conv3.norm.running_var loaded from res3_3_branch2c_bn_running_var of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.3.conv3.norm.weight loaded from res3_3_branch2c_bn_gamma of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res3.3.conv3.weight loaded from res3_3_branch2c_w of shape (512, 128, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.0.conv1.norm.bias loaded from res4_0_branch2a_bn_beta of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.0.conv1.norm.running_mean loaded from res4_0_branch2a_bn_running_mean of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.0.conv1.norm.running_var loaded from res4_0_branch2a_bn_running_var of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.0.conv1.norm.weight loaded from res4_0_branch2a_bn_gamma of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.0.conv1.weight loaded from res4_0_branch2a_w of shape (256, 512, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.0.conv2.norm.bias loaded from res4_0_branch2b_bn_beta of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.0.conv2.norm.running_mean loaded from res4_0_branch2b_bn_running_mean of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.0.conv2.norm.running_var loaded from res4_0_branch2b_bn_running_var of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.0.conv2.norm.weight loaded from res4_0_branch2b_bn_gamma of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.0.conv2.weight loaded from res4_0_branch2b_w of shape (256, 256, 3, 3)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.0.conv3.norm.bias loaded from res4_0_branch2c_bn_beta of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.0.conv3.norm.running_mean loaded from res4_0_branch2c_bn_running_mean of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.0.conv3.norm.running_var loaded from res4_0_branch2c_bn_running_var of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.0.conv3.norm.weight loaded from res4_0_branch2c_bn_gamma of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.0.conv3.weight loaded from res4_0_branch2c_w of shape (1024, 256, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.0.shortcut.norm.bias loaded from res4_0_branch1_bn_beta of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.0.shortcut.norm.running_mean loaded from res4_0_branch1_bn_running_mean of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.0.shortcut.norm.running_var loaded from res4_0_branch1_bn_running_var of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.0.shortcut.norm.weight loaded from res4_0_branch1_bn_gamma of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.0.shortcut.weight loaded from res4_0_branch1_w of shape (1024, 512, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.1.conv1.norm.bias loaded from res4_1_branch2a_bn_beta of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.1.conv1.norm.running_mean loaded from res4_1_branch2a_bn_running_mean of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.1.conv1.norm.running_var loaded from res4_1_branch2a_bn_running_var of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.1.conv1.norm.weight loaded from res4_1_branch2a_bn_gamma of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.1.conv1.weight loaded from res4_1_branch2a_w of shape (256, 1024, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.1.conv2.norm.bias loaded from res4_1_branch2b_bn_beta of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.1.conv2.norm.running_mean loaded from res4_1_branch2b_bn_running_mean of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.1.conv2.norm.running_var loaded from res4_1_branch2b_bn_running_var of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.1.conv2.norm.weight loaded from res4_1_branch2b_bn_gamma of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.1.conv2.weight loaded from res4_1_branch2b_w of shape (256, 256, 3, 3)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.1.conv3.norm.bias loaded from res4_1_branch2c_bn_beta of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.1.conv3.norm.running_mean loaded from res4_1_branch2c_bn_running_mean of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.1.conv3.norm.running_var loaded from res4_1_branch2c_bn_running_var of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.1.conv3.norm.weight loaded from res4_1_branch2c_bn_gamma of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.1.conv3.weight loaded from res4_1_branch2c_w of shape (1024, 256, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.2.conv1.norm.bias loaded from res4_2_branch2a_bn_beta of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.2.conv1.norm.running_mean loaded from res4_2_branch2a_bn_running_mean of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.2.conv1.norm.running_var loaded from res4_2_branch2a_bn_running_var of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.2.conv1.norm.weight loaded from res4_2_branch2a_bn_gamma of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.2.conv1.weight loaded from res4_2_branch2a_w of shape (256, 1024, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.2.conv2.norm.bias loaded from res4_2_branch2b_bn_beta of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.2.conv2.norm.running_mean loaded from res4_2_branch2b_bn_running_mean of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.2.conv2.norm.running_var loaded from res4_2_branch2b_bn_running_var of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.2.conv2.norm.weight loaded from res4_2_branch2b_bn_gamma of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.2.conv2.weight loaded from res4_2_branch2b_w of shape (256, 256, 3, 3)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.2.conv3.norm.bias loaded from res4_2_branch2c_bn_beta of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.2.conv3.norm.running_mean loaded from res4_2_branch2c_bn_running_mean of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.2.conv3.norm.running_var loaded from res4_2_branch2c_bn_running_var of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.2.conv3.norm.weight loaded from res4_2_branch2c_bn_gamma of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.2.conv3.weight loaded from res4_2_branch2c_w of shape (1024, 256, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.3.conv1.norm.bias loaded from res4_3_branch2a_bn_beta of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.3.conv1.norm.running_mean loaded from res4_3_branch2a_bn_running_mean of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.3.conv1.norm.running_var loaded from res4_3_branch2a_bn_running_var of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.3.conv1.norm.weight loaded from res4_3_branch2a_bn_gamma of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.3.conv1.weight loaded from res4_3_branch2a_w of shape (256, 1024, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.3.conv2.norm.bias loaded from res4_3_branch2b_bn_beta of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.3.conv2.norm.running_mean loaded from res4_3_branch2b_bn_running_mean of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.3.conv2.norm.running_var loaded from res4_3_branch2b_bn_running_var of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.3.conv2.norm.weight loaded from res4_3_branch2b_bn_gamma of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.3.conv2.weight loaded from res4_3_branch2b_w of shape (256, 256, 3, 3)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.3.conv3.norm.bias loaded from res4_3_branch2c_bn_beta of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.3.conv3.norm.running_mean loaded from res4_3_branch2c_bn_running_mean of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.3.conv3.norm.running_var loaded from res4_3_branch2c_bn_running_var of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.3.conv3.norm.weight loaded from res4_3_branch2c_bn_gamma of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.3.conv3.weight loaded from res4_3_branch2c_w of shape (1024, 256, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.4.conv1.norm.bias loaded from res4_4_branch2a_bn_beta of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.4.conv1.norm.running_mean loaded from res4_4_branch2a_bn_running_mean of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.4.conv1.norm.running_var loaded from res4_4_branch2a_bn_running_var of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.4.conv1.norm.weight loaded from res4_4_branch2a_bn_gamma of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.4.conv1.weight loaded from res4_4_branch2a_w of shape (256, 1024, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.4.conv2.norm.bias loaded from res4_4_branch2b_bn_beta of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.4.conv2.norm.running_mean loaded from res4_4_branch2b_bn_running_mean of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.4.conv2.norm.running_var loaded from res4_4_branch2b_bn_running_var of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.4.conv2.norm.weight loaded from res4_4_branch2b_bn_gamma of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.4.conv2.weight loaded from res4_4_branch2b_w of shape (256, 256, 3, 3)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.4.conv3.norm.bias loaded from res4_4_branch2c_bn_beta of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.4.conv3.norm.running_mean loaded from res4_4_branch2c_bn_running_mean of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.4.conv3.norm.running_var loaded from res4_4_branch2c_bn_running_var of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.4.conv3.norm.weight loaded from res4_4_branch2c_bn_gamma of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.4.conv3.weight loaded from res4_4_branch2c_w of shape (1024, 256, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.5.conv1.norm.bias loaded from res4_5_branch2a_bn_beta of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.5.conv1.norm.running_mean loaded from res4_5_branch2a_bn_running_mean of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.5.conv1.norm.running_var loaded from res4_5_branch2a_bn_running_var of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.5.conv1.norm.weight loaded from res4_5_branch2a_bn_gamma of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.5.conv1.weight loaded from res4_5_branch2a_w of shape (256, 1024, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.5.conv2.norm.bias loaded from res4_5_branch2b_bn_beta of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.5.conv2.norm.running_mean loaded from res4_5_branch2b_bn_running_mean of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.5.conv2.norm.running_var loaded from res4_5_branch2b_bn_running_var of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.5.conv2.norm.weight loaded from res4_5_branch2b_bn_gamma of shape (256,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.5.conv2.weight loaded from res4_5_branch2b_w of shape (256, 256, 3, 3)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.5.conv3.norm.bias loaded from res4_5_branch2c_bn_beta of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.5.conv3.norm.running_mean loaded from res4_5_branch2c_bn_running_mean of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.5.conv3.norm.running_var loaded from res4_5_branch2c_bn_running_var of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.5.conv3.norm.weight loaded from res4_5_branch2c_bn_gamma of shape (1024,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res4.5.conv3.weight loaded from res4_5_branch2c_w of shape (1024, 256, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.0.conv1.norm.bias loaded from res5_0_branch2a_bn_beta of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.0.conv1.norm.running_mean loaded from res5_0_branch2a_bn_running_mean of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.0.conv1.norm.running_var loaded from res5_0_branch2a_bn_running_var of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.0.conv1.norm.weight loaded from res5_0_branch2a_bn_gamma of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.0.conv1.weight loaded from res5_0_branch2a_w of shape (512, 1024, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.0.conv2.norm.bias loaded from res5_0_branch2b_bn_beta of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.0.conv2.norm.running_mean loaded from res5_0_branch2b_bn_running_mean of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.0.conv2.norm.running_var loaded from res5_0_branch2b_bn_running_var of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.0.conv2.norm.weight loaded from res5_0_branch2b_bn_gamma of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.0.conv2.weight loaded from res5_0_branch2b_w of shape (512, 512, 3, 3)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.0.conv3.norm.bias loaded from res5_0_branch2c_bn_beta of shape (2048,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.0.conv3.norm.running_mean loaded from res5_0_branch2c_bn_running_mean of shape (2048,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.0.conv3.norm.running_var loaded from res5_0_branch2c_bn_running_var of shape (2048,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.0.conv3.norm.weight loaded from res5_0_branch2c_bn_gamma of shape (2048,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.0.conv3.weight loaded from res5_0_branch2c_w of shape (2048, 512, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.0.shortcut.norm.bias loaded from res5_0_branch1_bn_beta of shape (2048,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.0.shortcut.norm.running_mean loaded from res5_0_branch1_bn_running_mean of shape (2048,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.0.shortcut.norm.running_var loaded from res5_0_branch1_bn_running_var of shape (2048,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.0.shortcut.norm.weight loaded from res5_0_branch1_bn_gamma of shape (2048,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.0.shortcut.weight loaded from res5_0_branch1_w of shape (2048, 1024, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.1.conv1.norm.bias loaded from res5_1_branch2a_bn_beta of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.1.conv1.norm.running_mean loaded from res5_1_branch2a_bn_running_mean of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.1.conv1.norm.running_var loaded from res5_1_branch2a_bn_running_var of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.1.conv1.norm.weight loaded from res5_1_branch2a_bn_gamma of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.1.conv1.weight loaded from res5_1_branch2a_w of shape (512, 2048, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.1.conv2.norm.bias loaded from res5_1_branch2b_bn_beta of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.1.conv2.norm.running_mean loaded from res5_1_branch2b_bn_running_mean of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.1.conv2.norm.running_var loaded from res5_1_branch2b_bn_running_var of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.1.conv2.norm.weight loaded from res5_1_branch2b_bn_gamma of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.1.conv2.weight loaded from res5_1_branch2b_w of shape (512, 512, 3, 3)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.1.conv3.norm.bias loaded from res5_1_branch2c_bn_beta of shape (2048,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.1.conv3.norm.running_mean loaded from res5_1_branch2c_bn_running_mean of shape (2048,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.1.conv3.norm.running_var loaded from res5_1_branch2c_bn_running_var of shape (2048,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.1.conv3.norm.weight loaded from res5_1_branch2c_bn_gamma of shape (2048,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.1.conv3.weight loaded from res5_1_branch2c_w of shape (2048, 512, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.2.conv1.norm.bias loaded from res5_2_branch2a_bn_beta of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.2.conv1.norm.running_mean loaded from res5_2_branch2a_bn_running_mean of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.2.conv1.norm.running_var loaded from res5_2_branch2a_bn_running_var of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.2.conv1.norm.weight loaded from res5_2_branch2a_bn_gamma of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.2.conv1.weight loaded from res5_2_branch2a_w of shape (512, 2048, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.2.conv2.norm.bias loaded from res5_2_branch2b_bn_beta of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.2.conv2.norm.running_mean loaded from res5_2_branch2b_bn_running_mean of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.2.conv2.norm.running_var loaded from res5_2_branch2b_bn_running_var of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.2.conv2.norm.weight loaded from res5_2_branch2b_bn_gamma of shape (512,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.2.conv2.weight loaded from res5_2_branch2b_w of shape (512, 512, 3, 3)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.2.conv3.norm.bias loaded from res5_2_branch2c_bn_beta of shape (2048,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.2.conv3.norm.running_mean loaded from res5_2_branch2c_bn_running_mean of shape (2048,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.2.conv3.norm.running_var loaded from res5_2_branch2c_bn_running_var of shape (2048,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.2.conv3.norm.weight loaded from res5_2_branch2c_bn_gamma of shape (2048,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.res5.2.conv3.weight loaded from res5_2_branch2c_w of shape (2048, 512, 1, 1)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.stem.conv1.norm.bias loaded from res_conv1_bn_beta of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.stem.conv1.norm.running_mean loaded from res_conv1_bn_running_mean of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.stem.conv1.norm.running_var loaded from res_conv1_bn_running_var of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.stem.conv1.norm.weight loaded from res_conv1_bn_gamma of shape (64,)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: backbone.bottom_up.stem.conv1.weight loaded from conv1_w of shape (64, 3, 7, 7)
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: Some model parameters or buffers are not in the checkpoint:
backbone.fpn_lateral2.{bias, weight}
backbone.fpn_lateral3.{bias, weight}
backbone.fpn_lateral4.{bias, weight}
backbone.fpn_lateral5.{bias, weight}
backbone.fpn_output2.{bias, weight}
backbone.fpn_output3.{bias, weight}
backbone.fpn_output4.{bias, weight}
backbone.fpn_output5.{bias, weight}
pixel_mean
pixel_std
proposal_generator.anchor_generator.cell_anchors.{0, 1, 2, 3, 4}
proposal_generator.rpn_head.anchor_deltas.{bias, weight}
proposal_generator.rpn_head.conv.{bias, weight}
proposal_generator.rpn_head.objectness_logits.{bias, weight}
roi_heads.box_head.fc1.{bias, weight}
roi_heads.box_head.fc2.{bias, weight}
roi_heads.box_predictor.bbox_pred.{bias, weight}
roi_heads.box_predictor.cls_score.{bias, weight}
roi_heads.mask_head.deconv.{bias, weight}
roi_heads.mask_head.mask_fcn1.{bias, weight}
roi_heads.mask_head.mask_fcn2.{bias, weight}
roi_heads.mask_head.mask_fcn3.{bias, weight}
roi_heads.mask_head.mask_fcn4.{bias, weight}
roi_heads.mask_head.predictor.{bias, weight}
roi_heads.mesh_head.stages.0.bottleneck.{bias, weight}
roi_heads.mesh_head.stages.0.gconvs.0.w0.{bias, weight}
roi_heads.mesh_head.stages.0.gconvs.0.w1.{bias, weight}
roi_heads.mesh_head.stages.0.gconvs.1.w0.{bias, weight}
roi_heads.mesh_head.stages.0.gconvs.1.w1.{bias, weight}
roi_heads.mesh_head.stages.0.gconvs.2.w0.{bias, weight}
roi_heads.mesh_head.stages.0.gconvs.2.w1.{bias, weight}
roi_heads.mesh_head.stages.0.verts_offset.{bias, weight}
roi_heads.mesh_head.stages.1.bottleneck.{bias, weight}
roi_heads.mesh_head.stages.1.gconvs.0.w0.{bias, weight}
roi_heads.mesh_head.stages.1.gconvs.0.w1.{bias, weight}
roi_heads.mesh_head.stages.1.gconvs.1.w0.{bias, weight}
roi_heads.mesh_head.stages.1.gconvs.1.w1.{bias, weight}
roi_heads.mesh_head.stages.1.gconvs.2.w0.{bias, weight}
roi_heads.mesh_head.stages.1.gconvs.2.w1.{bias, weight}
roi_heads.mesh_head.stages.1.verts_offset.{bias, weight}
roi_heads.mesh_head.stages.2.bottleneck.{bias, weight}
roi_heads.mesh_head.stages.2.gconvs.0.w0.{bias, weight}
roi_heads.mesh_head.stages.2.gconvs.0.w1.{bias, weight}
roi_heads.mesh_head.stages.2.gconvs.1.w0.{bias, weight}
roi_heads.mesh_head.stages.2.gconvs.1.w1.{bias, weight}
roi_heads.mesh_head.stages.2.gconvs.2.w0.{bias, weight}
roi_heads.mesh_head.stages.2.gconvs.2.w1.{bias, weight}
roi_heads.mesh_head.stages.2.verts_offset.{bias, weight}
roi_heads.voxel_head.deconv.{bias, weight}
roi_heads.voxel_head.predictor.{bias, weight}
roi_heads.voxel_head.voxel_fcn1.{bias, weight}
roi_heads.voxel_head.voxel_fcn2.{bias, weight}
roi_heads.voxel_head.voxel_fcn3.{bias, weight}
roi_heads.voxel_head.voxel_fcn4.{bias, weight}
roi_heads.z_head.z_fc1.{bias, weight}
roi_heads.z_head.z_fc2.{bias, weight}
roi_heads.z_head.z_pred.{bias, weight}
[05/27 09:32:55 d2.checkpoint.c2_model_loading]: The checkpoint state_dict contains keys that are not used by the model:
fc1000_b
fc1000_w
conv1_b
[05/27 09:32:55 d2.engine.train_loop]: Starting training from iteration 0
/python3.7/site-packages/torch/nn/functional.py:2506: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
[05/27 09:33:07 d2.utils.events]: eta: 8:51:53 iter: 19 total_loss: 6.434 loss_cls: 0.985 loss_box_reg: 0.000 loss_z_reg: 0.025 loss_mask: 0.691 loss_voxel: 2.066 loss_chamfer: 1.425 loss_normals: 0.272 loss_edge: 0.028 loss_rpn_cls: 0.661 loss_rpn_loc: 0.011 time: 0.3622 data_time: 0.0025 lr: 0.000293 max_mem: 1804M
[05/27 09:33:14 d2.utils.events]: eta: 8:45:29 iter: 39 total_loss: 5.395 loss_cls: 1.091 loss_box_reg: 0.002 loss_z_reg: 0.041 loss_mask: 0.672 loss_voxel: 2.006 loss_chamfer: 0.879 loss_normals: 0.266 loss_edge: 0.022 loss_rpn_cls: 0.517 loss_rpn_loc: 0.005 time: 0.3551 data_time: 0.0030 lr: 0.000338 max_mem: 1847M
[05/27 09:33:21 d2.utils.events]: eta: 8:41:40 iter: 59 total_loss: 4.499 loss_cls: 0.361 loss_box_reg: 0.000 loss_z_reg: 0.031 loss_mask: 0.664 loss_voxel: 1.893 loss_chamfer: 0.706 loss_normals: 0.287 loss_edge: 0.017 loss_rpn_cls: 0.443 loss_rpn_loc: 0.004 time: 0.3552 data_time: 0.0026 lr: 0.000383 max_mem: 1847M
[05/27 09:33:28 d2.utils.events]: eta: 8:45:59 iter: 79 total_loss: 4.963 loss_cls: 1.147 loss_box_reg: 0.005 loss_z_reg: 0.069 loss_mask: 0.562 loss_voxel: 1.653 loss_chamfer: 0.490 loss_normals: 0.277 loss_edge: 0.014 loss_rpn_cls: 0.264 loss_rpn_loc: 0.007 time: 0.3564 data_time: 0.0027 lr: 0.000428 max_mem: 1847M
[05/27 09:33:36 d2.utils.events]: eta: 8:46:12 iter: 99 total_loss: 4.247 loss_cls: 0.928 loss_box_reg: 0.002 loss_z_reg: 0.125 loss_mask: 0.641 loss_voxel: 1.622 loss_chamfer: 0.423 loss_normals: 0.279 loss_edge: 0.014 loss_rpn_cls: 0.332 loss_rpn_loc: 0.009 time: 0.3570 data_time: 0.0027 lr: 0.000473 max_mem: 1847M
[05/27 09:33:43 d2.utils.events]: eta: 8:48:47 iter: 119 total_loss: 3.844 loss_cls: 0.664 loss_box_reg: 0.001 loss_z_reg: 0.042 loss_mask: 0.692 loss_voxel: 1.499 loss_chamfer: 0.379 loss_normals: 0.290 loss_edge: 0.016 loss_rpn_cls: 0.168 loss_rpn_loc: 0.011 time: 0.3586 data_time: 0.0031 lr: 0.000518 max_mem: 1847M
[05/27 09:33:51 d2.utils.events]: eta: 8:49:35 iter: 139 total_loss: 3.737 loss_cls: 0.343 loss_box_reg: 0.003 loss_z_reg: 0.037 loss_mask: 0.652 loss_voxel: 1.469 loss_chamfer: 0.377 loss_normals: 0.281 loss_edge: 0.017 loss_rpn_cls: 0.250 loss_rpn_loc: 0.012 time: 0.3595 data_time: 0.0030 lr: 0.000563 max_mem: 1847M
[05/27 09:33:58 d2.utils.events]: eta: 8:47:39 iter: 159 total_loss: 3.685 loss_cls: 0.366 loss_box_reg: 0.001 loss_z_reg: 0.051 loss_mask: 0.630 loss_voxel: 1.322 loss_chamfer: 0.336 loss_normals: 0.269 loss_edge: 0.023 loss_rpn_cls: 0.163 loss_rpn_loc: 0.006 time: 0.3582 data_time: 0.0030 lr: 0.000608 max_mem: 1847M
[05/27 09:34:05 d2.utils.events]: eta: 8:47:03 iter: 179 total_loss: 3.930 loss_cls: 0.820 loss_box_reg: 0.001 loss_z_reg: 0.073 loss_mask: 0.665 loss_voxel: 1.418 loss_chamfer: 0.390 loss_normals: 0.266 loss_edge: 0.021 loss_rpn_cls: 0.167 loss_rpn_loc: 0.008 time: 0.3576 data_time: 0.0030 lr: 0.000653 max_mem: 1847M
[05/27 09:34:13 d2.utils.events]: eta: 8:45:46 iter: 199 total_loss: 3.188 loss_cls: 0.312 loss_box_reg: 0.004 loss_z_reg: 0.036 loss_mask: 0.683 loss_voxel: 1.212 loss_chamfer: 0.412 loss_normals: 0.272 loss_edge: 0.020 loss_rpn_cls: 0.145 loss_rpn_loc: 0.012 time: 0.3573 data_time: 0.0031 lr: 0.000698 max_mem: 1847M
[05/27 09:34:20 d2.utils.events]: eta: 8:46:48 iter: 219 total_loss: 3.581 loss_cls: 0.560 loss_box_reg: 0.001 loss_z_reg: 0.031 loss_mask: 0.688 loss_voxel: 1.376 loss_chamfer: 0.358 loss_normals: 0.268 loss_edge: 0.017 loss_rpn_cls: 0.114 loss_rpn_loc: 0.007 time: 0.3578 data_time: 0.0028 lr: 0.000743 max_mem: 1847M
[05/27 09:34:27 d2.utils.events]: eta: 8:48:04 iter: 239 total_loss: 3.891 loss_cls: 0.554 loss_box_reg: 0.001 loss_z_reg: 0.076 loss_mask: 0.686 loss_voxel: 1.411 loss_chamfer: 0.391 loss_normals: 0.269 loss_edge: 0.017 loss_rpn_cls: 0.194 loss_rpn_loc: 0.013 time: 0.3582 data_time: 0.0033 lr: 0.000788 max_mem: 1847M
[05/27 09:34:35 d2.utils.events]: eta: 8:48:51 iter: 259 total_loss: 3.795 loss_cls: 0.746 loss_box_reg: 0.000 loss_z_reg: 0.033 loss_mask: 0.619 loss_voxel: 1.444 loss_chamfer: 0.384 loss_normals: 0.273 loss_edge: 0.020 loss_rpn_cls: 0.111 loss_rpn_loc: 0.007 time: 0.3583 data_time: 0.0027 lr: 0.000833 max_mem: 1847M
[05/27 09:34:42 d2.utils.events]: eta: 8:49:45 iter: 279 total_loss: 3.436 loss_cls: 0.477 loss_box_reg: 0.001 loss_z_reg: 0.025 loss_mask: 0.630 loss_voxel: 1.409 loss_chamfer: 0.299 loss_normals: 0.259 loss_edge: 0.020 loss_rpn_cls: 0.184 loss_rpn_loc: 0.008 time: 0.3593 data_time: 0.0029 lr: 0.000878 max_mem: 1847M
[05/27 09:34:50 d2.utils.events]: eta: 8:50:32 iter: 299 total_loss: 3.401 loss_cls: 0.271 loss_box_reg: 0.001 loss_z_reg: 0.021 loss_mask: 0.690 loss_voxel: 1.445 loss_chamfer: 0.308 loss_normals: 0.260 loss_edge: 0.020 loss_rpn_cls: 0.169 loss_rpn_loc: 0.012 time: 0.3602 data_time: 0.0032 lr: 0.000923 max_mem: 1847M
[05/27 09:34:57 d2.utils.events]: eta: 8:51:34 iter: 319 total_loss: 3.619 loss_cls: 0.639 loss_box_reg: 0.001 loss_z_reg: 0.036 loss_mask: 0.653 loss_voxel: 1.305 loss_chamfer: 0.389 loss_normals: 0.261 loss_edge: 0.024 loss_rpn_cls: 0.113 loss_rpn_loc: 0.011 time: 0.3608 data_time: 0.0029 lr: 0.000968 max_mem: 1847M
[05/27 09:35:05 d2.utils.events]: eta: 8:51:32 iter: 339 total_loss: 3.426 loss_cls: 0.408 loss_box_reg: 0.000 loss_z_reg: 0.035 loss_mask: 0.557 loss_voxel: 1.452 loss_chamfer: 0.365 loss_normals: 0.260 loss_edge: 0.023 loss_rpn_cls: 0.124 loss_rpn_loc: 0.014 time: 0.3614 data_time: 0.0029 lr: 0.001013 max_mem: 1847M
[05/27 09:35:12 d2.utils.events]: eta: 8:51:42 iter: 359 total_loss: 3.023 loss_cls: 0.104 loss_box_reg: 0.000 loss_z_reg: 0.027 loss_mask: 0.672 loss_voxel: 1.329 loss_chamfer: 0.295 loss_normals: 0.276 loss_edge: 0.022 loss_rpn_cls: 0.147 loss_rpn_loc: 0.009 time: 0.3620 data_time: 0.0029 lr: 0.001058 max_mem: 1847M
[05/27 09:35:20 d2.utils.events]: eta: 8:52:28 iter: 379 total_loss: 3.988 loss_cls: 1.138 loss_box_reg: 0.001 loss_z_reg: 0.028 loss_mask: 0.610 loss_voxel: 1.446 loss_chamfer: 0.364 loss_normals: 0.260 loss_edge: 0.022 loss_rpn_cls: 0.159 loss_rpn_loc: 0.008 time: 0.3627 data_time: 0.0030 lr: 0.001103 max_mem: 1847M
[05/27 09:35:28 d2.utils.events]: eta: 8:54:13 iter: 399 total_loss: 4.119 loss_cls: 0.881 loss_box_reg: 0.001 loss_z_reg: 0.023 loss_mask: 0.675 loss_voxel: 1.350 loss_chamfer: 0.516 loss_normals: 0.278 loss_edge: 0.022 loss_rpn_cls: 0.228 loss_rpn_loc: 0.012 time: 0.3649 data_time: 0.0032 lr: 0.001148 max_mem: 2060M
[05/27 09:35:36 d2.utils.events]: eta: 8:54:42 iter: 419 total_loss: 3.727 loss_cls: 0.833 loss_box_reg: 0.005 loss_z_reg: 0.016 loss_mask: 0.660 loss_voxel: 1.360 loss_chamfer: 0.434 loss_normals: 0.277 loss_edge: 0.024 loss_rpn_cls: 0.132 loss_rpn_loc: 0.014 time: 0.3656 data_time: 0.0032 lr: 0.001193 max_mem: 2060M
[05/27 09:35:44 d2.utils.events]: eta: 8:56:32 iter: 439 total_loss: 3.678 loss_cls: 0.663 loss_box_reg: 0.003 loss_z_reg: 0.034 loss_mask: 0.669 loss_voxel: 1.301 loss_chamfer: 0.388 loss_normals: 0.267 loss_edge: 0.030 loss_rpn_cls: 0.184 loss_rpn_loc: 0.013 time: 0.3672 data_time: 0.0036 lr: 0.001238 max_mem: 2060M
[05/27 09:35:52 d2.utils.events]: eta: 8:58:30 iter: 459 total_loss: 3.903 loss_cls: 0.942 loss_box_reg: 0.013 loss_z_reg: 0.092 loss_mask: 0.642 loss_voxel: 1.426 loss_chamfer: 0.403 loss_normals: 0.267 loss_edge: 0.024 loss_rpn_cls: 0.210 loss_rpn_loc: 0.010 time: 0.3685 data_time: 0.0033 lr: 0.001283 max_mem: 2060M
[05/27 09:36:00 d2.utils.events]: eta: 9:01:04 iter: 479 total_loss: 4.350 loss_cls: 1.141 loss_box_reg: 0.003 loss_z_reg: 0.040 loss_mask: 0.658 loss_voxel: 1.431 loss_chamfer: 0.448 loss_normals: 0.270 loss_edge: 0.025 loss_rpn_cls: 0.151 loss_rpn_loc: 0.010 time: 0.3693 data_time: 0.0035 lr: 0.001328 max_mem: 2060M
[05/27 09:36:08 d2.utils.events]: eta: 9:02:57 iter: 499 total_loss: 4.393 loss_cls: 0.994 loss_box_reg: 0.002 loss_z_reg: 0.230 loss_mask: 0.629 loss_voxel: 1.287 loss_chamfer: 0.409 loss_normals: 0.278 loss_edge: 0.025 loss_rpn_cls: 0.155 loss_rpn_loc: 0.013 time: 0.3699 data_time: 0.0032 lr: 0.001373 max_mem: 2060M
[05/27 09:36:16 d2.utils.events]: eta: 9:06:46 iter: 519 total_loss: 4.648 loss_cls: 1.121 loss_box_reg: 0.002 loss_z_reg: 0.080 loss_mask: 0.679 loss_voxel: 1.270 loss_chamfer: 0.443 loss_normals: 0.272 loss_edge: 0.023 loss_rpn_cls: 0.111 loss_rpn_loc: 0.009 time: 0.3713 data_time: 0.0032 lr: 0.001418 max_mem: 2441M
[05/27 09:36:24 d2.utils.events]: eta: 9:07:17 iter: 539 total_loss: 4.536 loss_cls: 1.059 loss_box_reg: 0.011 loss_z_reg: 0.059 loss_mask: 0.633 loss_voxel: 1.260 loss_chamfer: 0.482 loss_normals: 0.267 loss_edge: 0.031 loss_rpn_cls: 0.170 loss_rpn_loc: 0.010 time: 0.3722 data_time: 0.0037 lr: 0.001463 max_mem: 2441M
[05/27 09:36:32 d2.utils.events]: eta: 9:07:55 iter: 559 total_loss: 3.435 loss_cls: 0.674 loss_box_reg: 0.006 loss_z_reg: 0.052 loss_mask: 0.684 loss_voxel: 1.338 loss_chamfer: 0.487 loss_normals: 0.249 loss_edge: 0.029 loss_rpn_cls: 0.178 loss_rpn_loc: 0.016 time: 0.3737 data_time: 0.0032 lr: 0.001508 max_mem: 2580M
[05/27 09:36:40 d2.utils.events]: eta: 9:07:53 iter: 579 total_loss: 3.800 loss_cls: 0.878 loss_box_reg: 0.001 loss_z_reg: 0.120 loss_mask: 0.667 loss_voxel: 1.255 loss_chamfer: 0.383 loss_normals: 0.253 loss_edge: 0.027 loss_rpn_cls: 0.180 loss_rpn_loc: 0.010 time: 0.3738 data_time: 0.0035 lr: 0.001553 max_mem: 2580M
[05/27 09:36:47 d2.utils.events]: eta: 9:07:45 iter: 599 total_loss: 3.184 loss_cls: 0.011 loss_box_reg: 0.004 loss_z_reg: 0.058 loss_mask: 0.668 loss_voxel: 1.127 loss_chamfer: 0.310 loss_normals: 0.262 loss_edge: 0.031 loss_rpn_cls: 0.153 loss_rpn_loc: 0.009 time: 0.3738 data_time: 0.0035 lr: 0.001598 max_mem: 2580M
[05/27 09:36:55 d2.utils.events]: eta: 9:08:13 iter: 619 total_loss: 4.273 loss_cls: 0.985 loss_box_reg: 0.001 loss_z_reg: 0.042 loss_mask: 0.660 loss_voxel: 1.138 loss_chamfer: 0.309 loss_normals: 0.255 loss_edge: 0.025 loss_rpn_cls: 0.113 loss_rpn_loc: 0.008 time: 0.3742 data_time: 0.0036 lr: 0.001643 max_mem: 2580M
[05/27 09:37:04 d2.utils.events]: eta: 9:10:09 iter: 639 total_loss: 4.578 loss_cls: 0.777 loss_box_reg: 0.005 loss_z_reg: 0.104 loss_mask: 0.680 loss_voxel: 1.445 loss_chamfer: 0.689 loss_normals: 0.256 loss_edge: 0.031 loss_rpn_cls: 0.203 loss_rpn_loc: 0.007 time: 0.3760 data_time: 0.0035 lr: 0.001688 max_mem: 2580M
[05/27 09:37:11 d2.utils.events]: eta: 9:09:01 iter: 659 total_loss: 3.606 loss_cls: 0.710 loss_box_reg: 0.001 loss_z_reg: 0.024 loss_mask: 0.670 loss_voxel: 1.000 loss_chamfer: 0.402 loss_normals: 0.244 loss_edge: 0.028 loss_rpn_cls: 0.121 loss_rpn_loc: 0.010 time: 0.3758 data_time: 0.0038 lr: 0.001733 max_mem: 2580M
[05/27 09:37:20 d2.utils.events]: eta: 9:10:34 iter: 679 total_loss: 3.512 loss_cls: 0.587 loss_box_reg: 0.009 loss_z_reg: 0.016 loss_mask: 0.668 loss_voxel: 1.157 loss_chamfer: 0.407 loss_normals: 0.245 loss_edge: 0.031 loss_rpn_cls: 0.147 loss_rpn_loc: 0.016 time: 0.3764 data_time: 0.0035 lr: 0.001778 max_mem: 2580M
[05/27 09:37:28 d2.utils.events]: eta: 9:10:41 iter: 699 total_loss: 3.698 loss_cls: 0.932 loss_box_reg: 0.007 loss_z_reg: 0.029 loss_mask: 0.650 loss_voxel: 1.151 loss_chamfer: 0.413 loss_normals: 0.236 loss_edge: 0.031 loss_rpn_cls: 0.150 loss_rpn_loc: 0.010 time: 0.3771 data_time: 0.0036 lr: 0.001823 max_mem: 2580M
[05/27 09:37:36 d2.utils.events]: eta: 9:10:42 iter: 719 total_loss: 3.112 loss_cls: 0.480 loss_box_reg: 0.001 loss_z_reg: 0.012 loss_mask: 0.641 loss_voxel: 1.083 loss_chamfer: 0.370 loss_normals: 0.237 loss_edge: 0.026 loss_rpn_cls: 0.112 loss_rpn_loc: 0.013 time: 0.3775 data_time: 0.0034 lr: 0.001868 max_mem: 2580M
[05/27 09:37:44 d2.utils.events]: eta: 9:12:03 iter: 739 total_loss: 4.250 loss_cls: 0.818 loss_box_reg: 0.001 loss_z_reg: 0.034 loss_mask: 0.652 loss_voxel: 1.375 loss_chamfer: 0.423 loss_normals: 0.247 loss_edge: 0.026 loss_rpn_cls: 0.181 loss_rpn_loc: 0.013 time: 0.3790 data_time: 0.0031 lr: 0.001913 max_mem: 2872M
[05/27 09:37:52 d2.utils.events]: eta: 9:11:40 iter: 759 total_loss: 3.782 loss_cls: 0.839 loss_box_reg: 0.001 loss_z_reg: 0.038 loss_mask: 0.687 loss_voxel: 1.133 loss_chamfer: 0.523 loss_normals: 0.251 loss_edge: 0.023 loss_rpn_cls: 0.201 loss_rpn_loc: 0.008 time: 0.3790 data_time: 0.0030 lr: 0.001958 max_mem: 2872M
[05/27 09:38:00 d2.utils.events]: eta: 9:12:18 iter: 779 total_loss: 3.657 loss_cls: 0.734 loss_box_reg: 0.000 loss_z_reg: 0.028 loss_mask: 0.694 loss_voxel: 1.103 loss_chamfer: 0.543 loss_normals: 0.257 loss_edge: 0.025 loss_rpn_cls: 0.211 loss_rpn_loc: 0.021 time: 0.3794 data_time: 0.0033 lr: 0.002003 max_mem: 2872M
[05/27 09:38:08 d2.utils.events]: eta: 9:13:09 iter: 799 total_loss: 3.595 loss_cls: 0.661 loss_box_reg: 0.000 loss_z_reg: 0.037 loss_mask: 0.630 loss_voxel: 1.137 loss_chamfer: 0.413 loss_normals: 0.253 loss_edge: 0.026 loss_rpn_cls: 0.160 loss_rpn_loc: 0.013 time: 0.3804 data_time: 0.0034 lr: 0.002048 max_mem: 2872M
[05/27 09:38:17 d2.utils.events]: eta: 9:14:38 iter: 819 total_loss: 3.308 loss_cls: 0.814 loss_box_reg: 0.000 loss_z_reg: 0.013 loss_mask: 0.664 loss_voxel: 1.059 loss_chamfer: 0.360 loss_normals: 0.244 loss_edge: 0.026 loss_rpn_cls: 0.104 loss_rpn_loc: 0.006 time: 0.3810 data_time: 0.0035 lr: 0.002093 max_mem: 2872M
[05/27 09:38:25 d2.utils.events]: eta: 9:14:53 iter: 839 total_loss: 3.427 loss_cls: 0.770 loss_box_reg: 0.000 loss_z_reg: 0.017 loss_mask: 0.630 loss_voxel: 1.149 loss_chamfer: 0.326 loss_normals: 0.242 loss_edge: 0.028 loss_rpn_cls: 0.108 loss_rpn_loc: 0.007 time: 0.3814 data_time: 0.0034 lr: 0.002138 max_mem: 2872M
[05/27 09:38:33 d2.utils.events]: eta: 9:15:49 iter: 859 total_loss: 3.110 loss_cls: 0.269 loss_box_reg: 0.001 loss_z_reg: 0.035 loss_mask: 0.613 loss_voxel: 1.031 loss_chamfer: 0.250 loss_normals: 0.247 loss_edge: 0.029 loss_rpn_cls: 0.230 loss_rpn_loc: 0.020 time: 0.3825 data_time: 0.0032 lr: 0.002183 max_mem: 2872M
[05/27 09:38:42 d2.utils.events]: eta: 9:16:40 iter: 879 total_loss: 3.691 loss_cls: 0.799 loss_box_reg: 0.000 loss_z_reg: 0.012 loss_mask: 0.652 loss_voxel: 1.213 loss_chamfer: 0.532 loss_normals: 0.254 loss_edge: 0.027 loss_rpn_cls: 0.152 loss_rpn_loc: 0.009 time: 0.3832 data_time: 0.0034 lr: 0.002228 max_mem: 2872M
[05/27 09:38:50 d2.utils.events]: eta: 9:16:46 iter: 899 total_loss: 3.549 loss_cls: 1.052 loss_box_reg: 0.001 loss_z_reg: 0.008 loss_mask: 0.657 loss_voxel: 1.023 loss_chamfer: 0.428 loss_normals: 0.244 loss_edge: 0.028 loss_rpn_cls: 0.124 loss_rpn_loc: 0.013 time: 0.3836 data_time: 0.0030 lr: 0.002273 max_mem: 2872M
[05/27 09:38:58 d2.utils.events]: eta: 9:16:39 iter: 919 total_loss: 3.120 loss_cls: 0.490 loss_box_reg: 0.000 loss_z_reg: 0.038 loss_mask: 0.598 loss_voxel: 1.139 loss_chamfer: 0.336 loss_normals: 0.250 loss_edge: 0.028 loss_rpn_cls: 0.099 loss_rpn_loc: 0.007 time: 0.3835 data_time: 0.0036 lr: 0.002318 max_mem: 2872M
[05/27 09:39:06 d2.utils.events]: eta: 9:17:14 iter: 939 total_loss: 3.500 loss_cls: 0.749 loss_box_reg: 0.001 loss_z_reg: 0.026 loss_mask: 0.651 loss_voxel: 1.169 loss_chamfer: 0.371 loss_normals: 0.248 loss_edge: 0.025 loss_rpn_cls: 0.111 loss_rpn_loc: 0.009 time: 0.3842 data_time: 0.0033 lr: 0.002363 max_mem: 2872M
[05/27 09:39:15 d2.utils.events]: eta: 9:18:22 iter: 959 total_loss: 3.912 loss_cls: 0.771 loss_box_reg: 0.001 loss_z_reg: 0.091 loss_mask: 0.630 loss_voxel: 1.263 loss_chamfer: 0.414 loss_normals: 0.261 loss_edge: 0.022 loss_rpn_cls: 0.264 loss_rpn_loc: 0.021 time: 0.3853 data_time: 0.0035 lr: 0.002408 max_mem: 2906M
[05/27 09:39:24 d2.utils.events]: eta: 9:18:35 iter: 979 total_loss: 3.480 loss_cls: 0.465 loss_box_reg: 0.063 loss_z_reg: 0.047 loss_mask: 0.668 loss_voxel: 1.266 loss_chamfer: 0.454 loss_normals: 0.249 loss_edge: 0.029 loss_rpn_cls: 0.135 loss_rpn_loc: 0.004 time: 0.3868 data_time: 0.0036 lr: 0.002453 max_mem: 2910M
[05/27 09:39:33 d2.utils.events]: eta: 9:19:31 iter: 999 total_loss: 4.672 loss_cls: 0.464 loss_box_reg: 0.008 loss_z_reg: 0.265 loss_mask: 0.660 loss_voxel: 1.259 loss_chamfer: 0.359 loss_normals: 0.237 loss_edge: 0.028 loss_rpn_cls: 0.242 loss_rpn_loc: 0.010 time: 0.3879 data_time: 0.0036 lr: 0.002498 max_mem: 2910M
Meshes contain nan or inf.
Meshes contain nan or inf.
Meshes contain nan or inf.
Traceback (most recent call last):
File "/meshrcnn/tools/train_net.py", line 113, in

main(args)

File "/meshrcnn/tools/train_net.py", line 100, in main
return trainer.train()
File "/detectron2-master/detectron2/engine/defaults.py", line 401, in train
super().train(self.start_iter, self.max_iter)
File "/detectron2-master/detectron2/engine/train_loop.py", line 132, in train
self.run_step()
File "/detectron2-master/detectron2/engine/train_loop.py", line 217, in run_step
self._detect_anomaly(losses, loss_dict)
File "/detectron2-master/detectron2/engine/train_loop.py", line 240, in _detect_anomaly
self.iter, loss_dict
FloatingPointError: Loss became infinite or NaN at iteration=1002!
loss_dict = {'loss_cls': tensor(nan, device='cuda:0', grad_fn=), 'loss_box_reg': tensor(nan, device='cuda:0', grad_fn=), 'loss_z_reg': tensor(nan, device='cuda:0', grad_fn=), 'loss_mask': tensor(1.0436e+10, device='cuda:0',
grad_fn=), 'loss_voxel': tensor(1.4543e+09, device='cuda:0', grad_fn=), 'loss_chamfer': 0, 'loss_normals': 0, 'loss_edge': 0, 'loss_rpn_cls': tensor(0.6048, device='cuda:0', grad_fn=), 'loss_rpn_loc': tensor(0.0055, device='cuda:0', grad_fn=)}
ERROR [05/27 09:39:34 d2.engine.train_loop]: Exception during training:
Traceback (most recent call last):
File "/detectron2-master/detectron2/engine/train_loop.py", line 132, in train
self.run_step()
File "/detectron2-master/detectron2/engine/train_loop.py", line 217, in run_step
self._detect_anomaly(losses, loss_dict)
File "/detectron2-master/detectron2/engine/train_loop.py", line 240, in _detect_anomaly
self.iter, loss_dict
FloatingPointError: Loss became infinite or NaN at iteration=1002!

loss_dict = {'loss_cls': tensor(nan, device='cuda:0', grad_fn=), 'loss_box_reg': tensor(nan, device='cuda:0', grad_fn=), 'loss_z_reg': tensor(nan, device='cuda:0', grad_fn=), 'loss_mask': tensor(1.0436e+10, device='cuda:0',
grad_fn=), 'loss_voxel': tensor(1.4543e+09, device='cuda:0', grad_fn=), 'loss_chamfer': 0, 'loss_normals': 0, 'loss_edge': 0, 'loss_rpn_cls': tensor(0.6048, device='cuda:0', grad_fn=), 'loss_rpn_loc': tensor(0.0055, device='cuda:0', grad_fn=)}
[05/27 09:39:34 d2.engine.hooks]: Overall training speed: 1000 iterations in 0:06:28 (0.3883 s / it)
[05/27 09:39:34 d2.engine.hooks]: Total training time: 0:06:33 (0:00:05 on hooks)

Hi, thank you for sharing your work！
My computer takes a lot of time loading unique model. So i just choose bed and bookcase to train. I adjust the json file and config file.
But when the train is in 1000 iterations, loss became infinite or NaN. Do you konw what's wrong with it? Or can you guide me how to solve this bug?

thanks,
czsc

Paper link

Mesh R-CNN: https://arxiv.org/abs/1906.02739

Matching the geometry to the input image

Hi there

I have a question about aligning the geometry to a camera in a 3D app like Maya.
Object classification/segmentation is ok example

The geometry obj file is created, but I was wondering if there is a way to align the geometry with a perspective camera that matches the original image.

• This geometry is in 3d space not aligned here
• The geometry is manually aligned here by eye here, but might not be accurate

I would like it to match the image - is there a relatively simple way this could be done?

Orthographic camera, t_z, focal_length

Hi @gkioxari,

1. In #22 you are talking about t_z true depth center per detected bbox, but i could not find this variable in the code - could you point us at it?

2. Not critical, but WRT Pix3D demo script - works great, although the reconstructions with larger focal_length values appear to come less distorted from my tests, e.g.:
   focal_length = 20. :
   

focal_lnegth = 1000.:

3. I'm looking for ways to apply Mesh R-CNN to work with orthographic camera. What would be your recommendations here regarding the corresponding code changes, and the K matrix (would you go just with a very large focal_length, or recommend replacing it with true ortho projection matrix?), etc.

thank you

Pre-processing Pix3D

Hi,
It is possible to get the code that generated the pix3d_s1_train.json pix3d_s2_train.json etc.
I'm curious about the K matrix, and how it was generated from the annotations provided with the pix3d dataset.

Thanks!

Question about the voxel data

I am not sure about how the voxel data is generated.
Is it generated using the obj file related to it?
During training, do you rotate the voxel predicted by the network?
And how do you determine the scale of the voxel?

Question about mesh_head loss

I can see from voxmesh_R50.yaml that for the shapenet you are using Chamfer and Edge losses only, while the Normal loss weight is set to 0.0.

1. Could you elaborate a little bit here on your weights decision?
2. Did you consider adding weighted Laplacian (mesh_laplacian_smoothing) loss to the mix? Why did you choose not to use it?
3. Did you consider using Pytorch3D differentiable renderer's silhouette-based loss? or do you think that Chamfer provides a better guidance here?

thank you!

function of the ROI_Z_Head

@gkioxari
thanks for the great repository.

when scrolling through the source code, I found the ROI head construction (meshrcnn/modelling/roi_heads).

In there, there's a z_head branch. I cannot figure out what this head branch is doing. Is it there to detect the z-coordinates of the mask?

thanks in advance.

detectron2 install failed

HI when the program install detectron2(pip install -e .)
It returns these:

Collecting detectron2 (from meshrcnn==1.0)
ERROR: Could not find a version that satisfies the requirement detectron2 (from meshrcnn==1.0) (from versions: none)
ERROR: No matching distribution found for detectron2 (from meshrcnn==1.0)

Does anyone have a solution?

Both ubuntu 16.04 and anaconda in windows have same problem.

Training with custom dataset.

Hi,
I tried with my dataset. And each image has multiple annotations. I provide the COCO format json file. But during the training I get the error said:

batch_crop_voels_within_box
     voxel_tensor = torch.stack(voxels.data, 0)
RuntimeError: invalid argument 0:

It seems that the error is because of the voxel data within the image have different size.
So my question is how to use your code to train the network that is provided with data that have multiple objects in the images.
Best regards,
zshyang

Issue: RuntimeError: Not compiled with GPU support.

Hi, I tried running the script after installing all the dependencies and received the following error. I have installed both pytorch3d as well as detectron2 correctly by following the instruction for cuda 10.1 but I don't know why I am still receiving this error.

(base) tasuka@tasuka-TM1701:~/Desktop/meshrcnn$ python3 demo/demo.py --config-file configs/pix3d/meshrcnn_R50_FPN.yaml --input ./input/sofa.jpg --output output_demo --onlyhighest MODEL.WEIGHTS meshrcnn://meshrcnn_R50.pth

[05/04 01:44:37 demo]: Arguments: Namespace(config_file='configs/pix3d/meshrcnn_R50_FPN.yaml', focal_length=20.0, input='./input/sofa.jpg', onlyhighest=True, opts=['MODEL.WEIGHTS', 'meshrcnn://meshrcnn_R50.pth'], output='output_demo') /pytorch/torch/csrc/utils/python_arg_parser.cpp:756: UserWarning: This overload of nonzero is deprecated:
nonzero(Tensor input, *, Tensor out)
Consider using one of the following signatures instead:
nonzero(Tensor input, *, bool as_tuple)

/pytorch/aten/src/ATen/native/BinaryOps.cpp:81: UserWarning: Integer division of tensors using div or / is deprecated, and in a future release div will perform true division as in Python 3. Use true_divide or floor_divide (// in Python) instead.

Traceback (most recent call last):
File "demo/demo.py", line 227, in <module>
predictions = demo.run_on_image(img, focal_length=args.focal_length)

File "demo/demo.py", line 59, in run_on_image
predictions = self.predictor(image)

File "/home/tasuka/Desktop/detectron2/detectron2/engine/defaults.py", line 218, in __call__
predictions = self.model([inputs])[0]

File "/home/tasuka/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
result = self.forward(*input, **kwargs)

File "/home/tasuka/Desktop/detectron2/detectron2/modeling/meta_arch/rcnn.py", line 108, in forward
return self.inference(batched_inputs)

File "/home/tasuka/Desktop/detectron2/detectron2/modeling/meta_arch/rcnn.py", line 170, in inference
results, _ = self.roi_heads(images, features, proposals, None)

File "/home/tasuka/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
result = self.forward(*input, **kwargs)

File "/home/tasuka/Desktop/meshrcnn/meshrcnn/modeling/roi_heads/roi_heads.py", line 164, in forward
pred_instances = self.forward_with_given_boxes(features, pred_instances)

File "/home/tasuka/Desktop/meshrcnn/meshrcnn/modeling/roi_heads/roi_heads.py", line 185, in forward_with_given_boxes
instances = self._forward_shape(features, instances)

File "/home/tasuka/Desktop/meshrcnn/meshrcnn/modeling/roi_heads/roi_heads.py", line 357, in _forward_shape
pred_meshes = self.mesh_head(mesh_features, init_mesh)

File "/home/tasuka/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
result = self.forward(*input, **kwargs)

File "/home/tasuka/Desktop/meshrcnn/meshrcnn/modeling/roi_heads/mesh_head.py", line 228, in forward
mesh, vert_feats = stage(x, mesh, vert_feats=vert_feats)

File "/home/tasuka/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
result = self.forward(*input, **kwargs)

File "/home/tasuka/Desktop/meshrcnn/meshrcnn/modeling/roi_heads/mesh_head.py", line 183, in forward
vert_feats_nopos = F.relu(graph_conv(vert_feats, mesh.edges_packed()))

File "/home/tasuka/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
result = self.forward(*input, **kwargs)

File "/home/tasuka/anaconda3/lib/python3.7/site-packages/pytorch3d/ops/graph_conv.py", line 71, in forward
neighbor_sums = gather_scatter(verts_w1, edges, self.directed)

File "/home/tasuka/anaconda3/lib/python3.7/site-packages/pytorch3d/ops/graph_conv.py", line 154, in forward
output = _C.gather_scatter(input, edges, directed, backward)

RuntimeError: Not compiled with GPU support.

meshrcnn_R50.pth Should be placed in that folder

meshrcnn_R50.pth Should be placed in that folder, I failed to download it when I run proxychains python demo/demo.py --config-file configs/pix3d/meshrcnn_R50_FPN.yaml --input /path/to/image --output output_demo --onlyhighest MODEL.WEIGHTS meshrcnn://meshrcnn_R50.pth often, it is not clear where to place it after downloading separately, thank you very much

Getting the Voxel outputs

I wish to tap into the network output in the middle. I want to retrieve the voxel representation before it is sent to the mesh refinement branch. Can anyone point me to as to where this is done/ how to do so?

Segmentation fault when running Demo

When running the demo I run into an segmentation fault. I've tried both with self downloaded weights as well as the default remote weights. Other than that I don't see things I could change.

Exact command I used:

python demo/demo.py \
--config-file configs/pix3d/meshrcnn_R50_FPN.yaml \
--input ../pf.jpg \
--output output_demo \
--onlyhighest meshrcnn_R50.pth

"Arguments to pooler must be lists"

Hi, thanks for sharing your code. I am following pix3d training steps, and I run into this error:

Traceback (most recent call last):
  File ".../detectron2/detectron2/engine/train_loop.py", line 132, in train
  File ".../detectron2/detectron2/engine/train_loop.py", line 212, in run_step
  File ".../anaconda3/envs/pytorch3d/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
  File ".../anaconda3/envs/pytorch3d/lib/python3.6/site-packages/torch/nn/parallel/distributed.py", line 447, in forward
  File ".../anaconda3/envs/pytorch3d/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
  File ".../detectron2/detectron2/modeling/meta_arch/rcnn.py", line 129, in forward
  File ".../anaconda3/envs/pytorch3d/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
  File ".../meshrcnn/meshrcnn/modeling/roi_heads/roi_heads.py", line 150, in forward
  File ".../detectron2/detectron2/modeling/roi_heads/roi_heads.py", line 619, in _forward_box
  File ".../anaconda3/envs/pytorch3d/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
  File ".../detectron2/detectron2/modeling/poolers.py", line 199, in forward
AssertionError: Arguments to pooler must be lists

I looked into it and found that the features that are passed in at line 150 of roi_heads.py is a dictionary that contains keys ['p2', 'p3', 'p4', 'p5', 'p6']. Which one is used for the paper? Any help would be much appreciated. Thanks.

Which shapenet voxel dataset is used?

In preprocess_shapenet.py, the file that is checked is 'model.presolid.binvox'. According to the shapenet documentation ( https://www.shapenet.org/download/shapenetcore ), these are not part of ShapeNetCore v1 or v2.

In R2N2's datasets, the models are called model.binvox.

Which dataset should we get for the shapenet voxelizations?

Pooler must be lists

Hi, I read about this essue, but it seems like new version fo ROI (d2) has some conficts with meshrcnn:
File "d:\detectron_projects\detectron2\detectron2\modeling\roi_heads\roi_heads.py", line 615, in _forward_box
box_features = self.box_pooler(features, [x.proposal_boxes for x in proposals])
File "C:\Users\aatul\anaconda3\envs\deto_env\lib\site-packages\torch\nn\modules\module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "d:\detectron_projects\detectron2\detectron2\modeling\poolers.py", line 199, in forward
), "Arguments to pooler must be lists"
AssertionError: Arguments to pooler must be lists

Coult you help?

How to demostrate aligned results

Hi, gkioxari. Thank you for your sharing, but could you tell me how to generate a figure like the last one in demo/README.md?