When I run 'CUDA_VISIBLE_DEVICES=1 ./tools/dist_train.sh configs/gfl_r50_1x.py 1 --validate', It raise 'TypeError: conv2d(): argument 'input' (position 1) must be Tensor, not DataContainer'. But i did not modify any source code. Why ?

hello! Support dense detection like anchor-based YOLOV3/YOLOV4? thanks

你好，作者：
QFL是一个很好的研究，是否可以指出QFL 在代码中的具体位置，感谢！

Dear team,
At the end of the first epoch I get:

2020-07-06 18:17:56,734 - mmdet - INFO - Epoch [1][300/359]     lr: 0.00732, eta: 1:58:24, time: 0.539, data_time: 0.006, memory: 4519, loss_qfl: 0.1841, loss_bbox: 0.4076, loss_dfl: 0.2519, loss: 0.8436
2020-07-06 18:18:23,969 - mmdet - INFO - Epoch [1][350/359]     lr: 0.00799, eta: 1:57:23, time: 0.545, data_time: 0.006, memory: 4519, loss_qfl: 0.1798, loss_bbox: 0.4058, loss_dfl: 0.2517, loss: 0.8373
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 

**508/506, 29.7 task/s, elapsed: 17s, ETA:     0s**

Traceback (most recent call last):
  File "./tools/train.py", line 151, in <module>
    main()
  File "./tools/train.py", line 147, in main
    meta=meta)
  File "/root/sharedfolder/production/GFocal/mmdet/apis/train.py", line 165, in train_detector
    runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
  File "/opt/conda/lib/python3.6/site-packages/mmcv/runner/runner.py", line 371, in run
    epoch_runner(data_loaders[i], **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/mmcv/runner/runner.py", line 285, in train
    self.call_hook('after_train_epoch')
  File "/opt/conda/lib/python3.6/site-packages/mmcv/runner/runner.py", line 238, in call_hook
    getattr(hook, fn_name)(self)
  File "/root/sharedfolder/production/GFocal/mmdet/core/evaluation/eval_hooks.py", line 76, in after_train_epoch
    self.evaluate(runner, results)
  File "/root/sharedfolder/production/GFocal/mmdet/core/evaluation/eval_hooks.py", line 33, in evaluate
    results, logger=runner.logger, **self.eval_kwargs)
  File "/root/sharedfolder/production/GFocal/mmdet/datasets/coco.py", line 326, in evaluate
    result_files, tmp_dir = self.format_results(results, jsonfile_prefix)
  File "/root/sharedfolder/production/GFocal/mmdet/datasets/coco.py", line 287, in format_results
    result_files = self.results2json(results, jsonfile_prefix)
  File "/root/sharedfolder/production/GFocal/mmdet/datasets/coco.py", line 217, in results2json
    json_results = self._det2json(results)
  File "/root/sharedfolder/production/GFocal/mmdet/datasets/coco.py", line 155, in _det2json
    data['category_id'] = self.cat_ids[label]
IndexError: list index out of range
Traceback (most recent call last):
  File "/opt/conda/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/opt/conda/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/opt/conda/lib/python3.6/site-packages/torch/distributed/launch.py", line 263, in <module>
    main()
  File "/opt/conda/lib/python3.6/site-packages/torch/distributed/launch.py", line 259, in main

This happens only when I pass the --validate option

./tools/dist_train.sh configs/gfl_x101_ms2x.py 4 --validate

Any advice about what may be wrong here?

Thanks,

Thank you for your great work!
I wonder whether you can share the idea about how to make statistics of IoU score and classification score to draw the correlation curve in figure 2.b of your paper?

If it's possible, could you share the code?
Thank you so much!

Why I set the task in "test", it doesnot get the test results.

Hi,

I'm very impressed by your excellent work.

When I read and compared your experiments about FCOS and ATSS, I was confused by those values you mentioned in Table 2 (a).

Could you explain what I missed?

Thanks

Hi, thanks a lot for your great work and I am sorry to bother you but there is a question that nags me a lot.
As mentioned in your paper, the supervision of the localization quality estimation is currently assigned for positive samples only which is unreliable as negatives may get chances to have uncontrollably higher quality predictions.
Merging quality prediction into classification score is fantastic. But here is the thing: Have your tried to keep the quality branch and train it with the inclusion of negative samples with proposed QFL?
I've tried this in FCOS and it turned out that centerness predictions for positive sample in minival were pretty bad. And I've taken those super parameters like initial bias prior for centerness branch into consideration but results were always bad.
Hope to hear your insights.
Thanks again.
@implus

Hello
How are you?
Thanks for contributing this project.
I have a question.
Does this project support class-incremental training for saving training time without catastrophic forgetting?
Let's suppose I've trained a model on the dataset with K classes for 7 days.
If a new class is added into this dataset, should we train a model with the expanded dataset (K+1 classes) from begin?
If so, it is so expensive, especially in case of object detection in retail store.
That's because a new class of good is added frequently in retail store.
Can we train a new model in short time with the original weight on the expanded dataset?
I think that this is a very important function.
I send some recommended papers for this project.
https://arxiv.org/pdf/1708.06977.pdf
https://arxiv.org/pdf/2003.04668.pdf
https://arxiv.org/pdf/2003.06957.pdf
https://arxiv.org/pdf/2003.07304.pdf
Please let me know if you have a willing to implement this.
Thanks

Thanks for your wonderful job. I find that you donot use alpha(which is 0.25/0.75 for pos/negative samples in focal loss), have you tried it in QFL?

Hi, I am really interested in the GFL work. I would want to know your implementation detail about GFL in FCOS. I have tested the QFL in FCOS, but I couldn't get the results in paper. FCOS_R_50_1x with it's ctr. sampling, GIOU loss, Normalization improvements can get 39.2 mAP in Detectron2. It's the same as ATSS with centerness. But when I remove the centerness branch and multiply the centerness ground truth with class score as soft label, using QFL, the result is 38.59 mAP. Could you help me to find out the issue?

Thanks!

QFL的描述很相似的感觉，在heatmap的中心点周围去增加惩罚

Have you met this problem? And how to solve it

the article says \Delta means the interval between y_i and y_{i+1}.

In glocal_loss.py
dis_left = label.long() dis_right = dis_left + 1

I consider that 1 means the interval. Because F.cross_entropy needs an integer number to be the true label.
I find that there has a \Delta equals to 0.5 in your ablation study.(Table 2 (c))
So what is the setting of your CE loss when \Delta equals to 0.5?

我目前使用的是focal loss，如何更改能直接使用您提出的GFL,来进行实验？

Can somebody answer something on this

Hi~I have done an experiment on centernet by changing original penalty focal loss to QFL. But my training is continued oscillating and cannot converge. I have checked my code and cannot find any obvious bugs. Do you have any suggestions? Thanks a lot.

I set the number of GPUs to 1, and then the predicted quality values of iou all become 0, making the training unable to proceed normally. Whether it can be improved without increasing the GPUs
Thanks!

norm_anchor_center = self.anchor_center(pos_anchors) / stride

pos_bbox_pred_distance = self.distribution_project(pos_bbox_pred)

pos_decode_bbox_pred = distance2bbox(norm_anchor_center,
                             pos_bbox_pred_distance)
pos_decode_bbox_targets = pos_bbox_targets / stride

target_ltrb = bbox2distance(norm_anchor_center,
                            pos_decode_bbox_targets, 
                            self.reg_max).reshape(-1)
score[pos_inds] = self.iou_target(pos_decode_bbox_pred.detach(),
                            pos_decode_bbox_targets)

请问在计算iou的时候，为什么norm_anchor_center和pos_bbox_targets要除以stride呢？计算iou通常不都是在原图上计算吗？

Looking at the code I faced with pos and pos_inds but I could not understand what they are and what they do.

Hello,
I tried the link on Mmdetection however not only the names of the config files are different but also the Links to the weights are dead.
For instance:
https://open-mmlab.s3.ap-northeast-2.amazonaws.com/mmdetection/v2.0/gfl/gfl_x101_32x4d_fpn_mstrain_2x_coco/gfl_x101_32x4d_fpn_mstrain_2x_coco_20200630_102002-50c1ffdb.pth

Hi guys, does anyone know the total training time of GFocal?

If i have a C++ code,and i want use GFL ,how can i use GFL(mmdetection) by C++.
Using Pytorch? or?

如题，GFL可能会带来更高的AP，但是误报率呢
我用Nanodet训练自己的数据集，AP确实比yolov3高很多，但是误报率很高，猜想和GFL的置信度表达有关，可否解下疑惑，多谢

It's easy to get the classification-IoU score in inference, but how can I get its ground truth in training? while generating the ground truth, I do not has a model to get predict bbox, but only has the ground truth of bbox, so can not get the iou to be the ground truth of classification-IoU score.

When I try to train with fp16 precision I get the following errors:

Traceback (most recent call last):
File "tools/train.py", line 153, in
main()
File "tools/train.py", line 149, in main
meta=meta)
File "/home/mmdetection/mmdet/apis/train.py", line 128, in train_detector
runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
File "/home/schlmi/miniconda2/envs/open-mmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 122, in run
epoch_runner(data_loaders[i], **kwargs)
File "/home/schlmi/miniconda2/envs/open-mmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 32, in train
**kwargs)
File "/home/schlmi/miniconda2/envs/open-mmlab/lib/python3.7/site-packages/mmcv/parallel/data_parallel.py", line 31, in train_step
return self.module.train_step(*inputs[0], **kwargs[0])
File "/home/schlmi/mmdetection/mmdet/models/detectors/base.py", line 236, in train_step
losses = self(**data)
File "/home/schlmi/miniconda2/envs/open-mmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/schlmi/mmdetection/mmdet/core/fp16/decorators.py", line 77, in new_func
output = old_func(*new_args, **new_kwargs)
File "/home/schlmi/mmdetection/mmdet/models/detectors/base.py", line 171, in forward
return self.forward_train(img, img_metas, **kwargs)
File "/home/schlmi/research/dudoof/mmdetection/mmdet/models/detectors/single_stage.py", line 93, in forward_train
gt_labels, gt_bboxes_ignore)
File "/home/schlmi/mmdetection/mmdet/models/dense_heads/base_dense_head.py", line 54, in forward_train
losses = self.loss(*loss_inputs, gt_bboxes_ignore=gt_bboxes_ignore)
File "/home/schlmi/mmdetection/mmdet/core/fp16/decorators.py", line 156, in new_func
output = old_func(*new_args, **new_kwargs)
File "/home/schlmi/mmdetection/mmdet/models/dense_heads/gfl_head.py", line 369, in loss
num_total_samples=num_total_samples)
File "/home/schlmi/mmdetection/mmdet/core/utils/misc.py", line 54, in multi_apply
return tuple(map(list, zip(*map_results)))
File "/home/schlmi/mmdetection/mmdet/models/dense_heads/gfl_head.py", line 266, in loss_single
pos_bbox_pred_corners = self.integral(pos_bbox_pred)
File "/home/schlmi/miniconda2/envs/open-mmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/schlmi/mmdetection/mmdet/models/dense_heads/gfl_head.py", line 55, in forward
x = F.linear(x, self.project).reshape(-1, 4)
File "/home/schlmi/miniconda2/envs/open-mmlab/lib/python3.7/site-packages/torch/nn/functional.py", line 1372, in linear
output = input.matmul(weight.t())
RuntimeError: Expected object of scalar type Float but got scalar type Half for argument #2 'vec' in call to _th_mv

Is there a way to fix this?

Hi,
Thank you for sharing the great work.
I have few questions listed below.

1. Table 1(a) in paper, you replaced the centerness branch with iou branch in FCOS, and got some quality gain. I'm curious about the details about this part. I do some experiment myself and found the training is not working. In the original FCOS, the regression loss is weighted by centerness target, and if I simply replace the centerness target with the iou target, the regression loss and iou loss will all be zero, so the training failed. It will be very nice if you share some code or details about this experiment.
2. In gfl_head.py, the loss_dfl and loss_bbox is weighted by "predicted score" which is different in FCOS and ATSS. Is there any reason about this part?
   Thank you in advance.

Can I get the quality scores only with the logits ?

https://github.com/implus/GFocal/blob/master/mmdet/models/losses/gfocal_loss.py#L42-L46
`

disl = label.long()
disr = disl + 1

wl = disr.float() - label # this is (label.long() + 1).float() - label , which is simply 1.0
wr = label - disl.float() # this is label - label.long().float(), which is 0.0

`
therefore, your GFL is simply CE?

Welcome update to OpenMMLab 2.0

I am Vansin, the technical operator of OpenMMLab. In September of last year, we announced the release of OpenMMLab 2.0 at the World Artificial Intelligence Conference in Shanghai. We invite you to upgrade your algorithm library to OpenMMLab 2.0 using MMEngine, which can be used for both research and commercial purposes. If you have any questions, please feel free to join us on the OpenMMLab Discord at https://discord.gg/amFNsyUBvm or add me on WeChat (van-sin) and I will invite you to the OpenMMLab WeChat group.

Here are the OpenMMLab 2.0 repos branches:

Attention: please create a new virtual environment for OpenMMLab 2.0.

I am training model on custom data and using config file gfl_dcn_r101_ms2x.py. After 4 epochs I am getting following error -

File "tools/train.py", line 151, in <module> main() File "tools/train.py", line 147, in main meta=meta) File "/usr/local/lib/python3.6/dist-packages/mmdet-1.1.0+32863f2-py3.6-linux-x86_64.egg/mmdet/apis/train.py", line 165, in train_detector runner.run(data_loaders, cfg.workflow, cfg.total_epochs) File "/usr/local/lib/python3.6/dist-packages/mmcv/runner/runner.py", line 359, in run epoch_runner(data_loaders[i], **kwargs) File "/usr/local/lib/python3.6/dist-packages/mmcv/runner/runner.py", line 263, in train self.model, data_batch, train_mode=True, **kwargs) File "/usr/local/lib/python3.6/dist-packages/mmdet-1.1.0+32863f2-py3.6-linux-x86_64.egg/mmdet/apis/train.py", line 75, in batch_processor losses = model(**data) File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 550, in __call__ result = self.forward(*input, **kwargs) File "/usr/local/lib/python3.6/dist-packages/torch/nn/parallel/data_parallel.py", line 153, in forward return self.module(*inputs[0], **kwargs[0]) File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 550, in __call__ result = self.forward(*input, **kwargs) File "/usr/local/lib/python3.6/dist-packages/mmdet-1.1.0+32863f2-py3.6-linux-x86_64.egg/mmdet/core/fp16/decorators.py", line 49, in new_func return old_func(*args, **kwargs) File "/usr/local/lib/python3.6/dist-packages/mmdet-1.1.0+32863f2-py3.6-linux-x86_64.egg/mmdet/models/detectors/base.py", line 147, in forward return self.forward_train(img, img_metas, **kwargs) File "/usr/local/lib/python3.6/dist-packages/mmdet-1.1.0+32863f2-py3.6-linux-x86_64.egg/mmdet/models/detectors/single_stage.py", line 71, in forward_train *loss_inputs, gt_bboxes_ignore=gt_bboxes_ignore) File "/usr/local/lib/python3.6/dist-packages/mmdet-1.1.0+32863f2-py3.6-linux-x86_64.egg/mmdet/core/fp16/decorators.py", line 127, in new_func return old_func(*args, **kwargs) File "/usr/local/lib/python3.6/dist-packages/mmdet-1.1.0+32863f2-py3.6-linux-x86_64.egg/mmdet/models/anchor_heads/gfl_head.py", line 254, in loss cfg=cfg) File "/usr/local/lib/python3.6/dist-packages/mmdet-1.1.0+32863f2-py3.6-linux-x86_64.egg/mmdet/core/utils/misc.py", line 24, in multi_apply return tuple(map(list, zip(*map_results))) File "/usr/local/lib/python3.6/dist-packages/mmdet-1.1.0+32863f2-py3.6-linux-x86_64.egg/mmdet/models/anchor_heads/gfl_head.py", line 186, in loss_single avg_factor=1.0) File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 550, in __call__ result = self.forward(*input, **kwargs) File "/usr/local/lib/python3.6/dist-packages/mmdet-1.1.0+32863f2-py3.6-linux-x86_64.egg/mmdet/models/losses/iou_loss.py", line 200, in forward return (pred * weight).sum() # 0 RuntimeError: The size of tensor a (4) must match the size of tensor b (529) at non-singleton dimension 1
How to resolve this?

I want to check the improved representation of the trained model with GFL, so I want to draw the distribution of feature map like Fig.3.

What should I do？

By the way, I think the plot is just for visulization to make readers understand better. Am I right?

代码里面好像没有这个文件