hustvl / bmaskr-cnn

[ECCV 2020] Boundary-preserving Mask R-CNN

HUSTVL Lab Website

Visit hustvl.github.io 🚀

Built with Lab Website Template

• Lab Website Template v1.1.6
• Template Docs
• Jekyll
• Material Design Colors
• Font Awesome Icons

Installation

https://jekyllrb.com/docs/installation/ubuntu/

Install needed Ruby packages by running bundle install.

Usage

Preview the cite by running

bundle exec jekyll serve --open-url --livereload --trace

bmaskr-cnn's Issues

How to test customized images on CPU?

Dear Author,

Thank you for your great work and open source code. I wonder how can I test my own images without GPU on your pre-trained images. Which part should I change based on your code? As I need to test images on the PC without GPU in the near future.

Looking forward to your reply. Thanks again.

other datasets

I would like to know how to train and test other datasets with bmask-rcnn network？

visualize

I need to visualize how the segmentation results are processed

I training my datasets and I have 3 classes,but when I testing ,it has a problem?

Hello!
I have a problem when I test my data.
The problem is that:
Skip loading parameter 'roi_heads.box_predictor.cls_score.weight' to the model due to incompatible shapes: (81, 1024) in the checkpoint but (4, 1024) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.cls_score.bias' to the model due to incompatible shapes: (81,) in the checkpoint but (4,) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.bbox_pred.weight' to the model due to incompatible shapes: (320, 1024) in the checkpoint but (12, 1024) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.bbox_pred.bias' to the model due to incompatible shapes: (320,) in the checkpoint but (12,) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.mask_head.mask_predictor.weight' to the model due to incompatible shapes: (80, 256, 1, 1) in the checkpoint but (3, 256, 1, 1) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.mask_head.mask_predictor.bias' to the model due to incompatible shapes: (80,) in the checkpoint but (3,) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.mask_head.boundary_predictor.weight' to the model due to incompatible shapes: (80, 256, 1, 1) in the checkpoint but (3, 256, 1, 1) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.mask_head.boundary_predictor.bias' to the model due to incompatible shapes: (80,) in the checkpoint but (3,) in the model! You might want to double check if this is expected.

I set the cfg.MODEL.RETINANET.NUM_CLASSES = 3 about set_up() and_C.MODEL.RETINANET.NUM_CLASSES = 3,_C.MODEL.ROI_HEADS.NUM_CLASSES = 3 about detectron2/config/defaults.py ,what's wrong about it ,or Where I do not set it to do，please tell me ,thank you very much!

If I want to transfer to caffe2, how can I use the official documentation

how to predict "image" with pretrained weight?

Hello,

I usually use detectron2 and predict image detection and classfication. I tried to test you solution with almost same method used in detection2. but I got test code and error like below

---------------------------------------TEST Code------------------------------------------------------
cfg = get_cfg()
bmaskrcnn.add_boundary_preserving_config(cfg)
// Load a config from file
cfg.merge_from_file("bmask_repo/projects/BMaskR-CNN/configs/bmask_rcnn_R_50_FPN_1x.yaml")
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5
cfg.MODEL.WEIGHTS = "/content/gdrive/My Drive/bmask/bmask_rcnn_r50_1x.pth"
bmask_predictor = DefaultPredictor(cfg)
outputs = bmask_predictor(im)

--------------------------------------ERROR MESSAGE-------------------------------------------------------
TypeError Traceback (most recent call last)
in ()
12 #cfg.MODEL.WEIGHTS = "https://dl.fbaipublicfiles.com/detectron2/PointRend/InstanceSegmentation/pointrend_rcnn_R_50_FPN_1x_cityscapes/164255101/model_final_318a02.pkl"
13 #cfg.MODEL.WEIGHTS = "https://dl.fbaipublicfiles.com/detectron2/PointRend/SemanticSegmentation/pointrend_semantic_R_101_FPN_1x_cityscapes/186480235/model_final_5f3665.pkl"
---> 14 bmask_predictor = DefaultPredictor(cfg)
15 outputs = bmask_predictor(im)

5 frames
/usr/local/lib/python3.6/dist-packages/detectron2/modeling/roi_heads/roi_heads.py in init(self, cfg, input_shape)
469 super(StandardROIHeads, self).init(cfg, input_shape)
470 self._init_box_head(cfg)
--> 471 self._init_mask_head(cfg)
472 self._init_keypoint_head(cfg)
473

TypeError: _init_mask_head() missing 1 required positional argument: 'input_shape'

Could you give me guide to test ? if you have colab code, I am more happy .

作者好，训练测试还不能运行吧？指定的.yaml文件不存在

作者好，训练测试还不能运行吧？指定的.yaml文件不存在，期待早日发布

FloatingPointError: Predicted boxes or scores contain Inf/NaN. Training has diverged.

Hi, when I run configs/bmask_rcnn_R_50_FPN_1x.yaml, it ran 739 iters,then I meet the question as follows,
Traceback (most recent call last):
File "/home/l547/anaconda3/envs/zt/lib/python3.7/site-packages/detectron2/engine/train_loop.py", line 140, in train
self.run_step()
File "/home/l547/anaconda3/envs/zt/lib/python3.7/site-packages/detectron2/engine/train_loop.py", line 226, in run_step
loss_dict = self.model(data)
File "/home/l547/anaconda3/envs/zt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/home/l547/anaconda3/envs/zt/lib/python3.7/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 160, in forward
proposals, proposal_losses = self.proposal_generator(images, features, gt_instances)
File "/home/l547/anaconda3/envs/zt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/home/l547/anaconda3/envs/zt/lib/python3.7/site-packages/detectron2/modeling/proposal_generator/rpn.py", line 449, in forward
anchors, pred_objectness_logits, pred_anchor_deltas, images.image_sizes
File "/home/l547/anaconda3/envs/zt/lib/python3.7/site-packages/detectron2/modeling/proposal_generator/rpn.py", line 486, in predict_proposals
self.training,
File "/home/l547/anaconda3/envs/zt/lib/python3.7/site-packages/detectron2/modeling/proposal_generator/proposal_utils.py", line 92, in find_top_rpn_proposals
"Predicted boxes or scores contain Inf/NaN. Training has diverged."
FloatingPointError: Predicted boxes or scores contain Inf/NaN. Training has diverged.

I want to know how to address,please?

I want the original image in Figure 2 of the paper. Can you upload it to GitHub?

I want the original image in Figure 2 of the paper. Can you upload it to GitHub?

How Can I train BMaskR-CNN on MOT20 dataset?

What is different between Cascade BMask R-CNN and X101-FPN in detectron2 ?

Hello,

I have question about your work.
It seems X101-FPN (detectron2's result table) is better than R-101.
If your model use X101-FPN baseline, is it much better? So if you didn't, is there some reason?

Effect of longer training schedules (3x) and ResNeXt 101 and 152 backbones?

Hi,
Thank you for your good work. I have a couple of questions:

1. Did you guys test this network for longer training schedules and see if the difference between regular MaskRCNN holds ? PointRend paper shows that even if used 3x training schedule they still get 1.3%mAP better than regular MaskRCNN.

2. On the similar lines, were you able to show that the performance difference between a regular MaskRCNN and a BMaskRCNN hold if you use a higher capacity network say, ResNeXt 101 and ResNeXt 152?

use caffe2_converter.py change the model to caffe2

the error is :

File "F:\BMaskR-CNN\projects\BMaskR-CNN\bmaskrcnn\mask_head.py", line 278, in forward
mask_rcnn_inference(mask_logits, instances)
File "f:\bmaskr-cnn\detectron2\modeling\roi_heads\mask_head.py", line 146, in mask_rcnn_inference
mask_probs_pred = pred_mask_logits[indices, class_pred][:, None].sigmoid()
IndexError: tensors used as indices must be long, byte or bool tensors

than change the mask_head.py ;
def mask_rcnn_inference(pred_mask_logits, pred_instances):
class_pred = cat([i.pred_classes for i in pred_instances]).type(torch.long)

the error is ：
return cm_exit(cm, *exc_details)
File "D:\Anaconda3\envs\NBMASK\lib\contextlib.py", line 88, in exit
next(self.gen)
File "f:\bmaskr-cnn\detectron2\export\patcher.py", line 101, in mock_mask_rcnn_inference
assert mocked_func.call_count > 0
AssertionError

I want to know how to change myself model to caffe2 model.

Cityscapes model link seems to be wrong

This is what I get by clicking the Cityscapes model line, should it be a R50 model?
Could you check if you put the correct link?

Thanks.

n/a

solved

Where to put the training or testing dataset?

Dear Author,

I wonder where to put the training and testing dataset? Are there any rules to name them. It is weired that you did not have the parameters in your parameter lines. Do I need to change the dataset name in the code? Hoping for your reply. Thanks!

how can I test a pre trained model on my custom images ?

where can I find the bmask_rcnn_R_50_FPN_1x.py file?

Thanks for sharing the nice code.
where can I find the bmask_rcnn_R_50_FPN_1x.py file?

边缘算子，请问数据集需要单独把边缘标注吗，还是就用分割的mask，然后模型自动用Laplacian算子来生成soft边界

Train with custom dataset(cocoFormat), meet KeyError: 'Non-existent config key: MODEL.BOUNDARY_MASK_HEAD'

File "//trainCode/segmentation/BMaskR-CNN-master/projects/BMaskR-CNN/train_yjh.py", line 217, in setup
cfg.merge_from_file(args.config_file)
File "//trainCode/segmentation/BMaskR-CNN-master/detectron2/config/config.py", line 49, in merge_from_file
self.merge_from_other_cfg(loaded_cfg)
File "/usr/local/lib/python3.6/dist-packages/fvcore/common/config.py", line 124, in merge_from_other_cfg
return super().merge_from_other_cfg(cfg_other)
File "/usr/local/lib/python3.6/dist-packages/yacs/config.py", line 217, in merge_from_other_cfg
_merge_a_into_b(cfg_other, self, self, [])
File "/usr/local/lib/python3.6/dist-packages/yacs/config.py", line 478, in _merge_a_into_b
_merge_a_into_b(v, b[k], root, key_list + [k])
File "/usr/local/lib/python3.6/dist-packages/yacs/config.py", line 491, in _merge_a_into_b
raise KeyError("Non-existent config key: {}".format(full_key))
KeyError: 'Non-existent config key: MODEL.BOUNDARY_MASK_HEAD'

How to solve the problem?

How do I convert my model to ONNX，I made an error in the conversion

is boundary only used for train not for inference?

i don't found boundary used in inference. is the boundary just used for training?

Pretrained model available?

Dear author,
Thank you very much for sharing your great work! I hope to know are there pretrained bmaskr-cnn models on Coco Dataset? I saw "COCO-InstanceSegmentation/bmask_rcnn_R_50_FPN_1x.yaml": "137260431/model_final_a54504.pkl" in the model_zoo.py. However, it cann't be downloaded.
Your reply is greatly appreciated. Thank you very much!

large occluded objects

I found a problem, it is that there are several large occluded objects in my test set, and the model I trained cannot recognize and segment them.

error in prediction:TypeError: argument of type 'GeneralizedRCNN' is not iterable

error occurs in this line of code:
predictor = DefaultPredictor(cfg)

the error report is as followsL
/usr/local/lib/python3.7/dist-packages/detectron2/checkpoint/detection_checkpoint.py in _load_file(self, filename)
87
88 loaded = super()._load_file(filename) # load native pth checkpoint
---> 89 if "model" not in loaded:
90 loaded = {"model": loaded}
91 return loaded

TypeError: argument of type 'GeneralizedRCNN' is not iterable

Thanks.

KeyError: "No object named 'BoundaryROIHeads' found in 'ROI_HEADS' registry!"

When trying to run the demo.py, keep getting registry related problems.
Have tried to modify the code in the detectron2 module, but unsuccessfully.

Is there anyone who encountered this problem? How did you solve it?