deepsceneseg / efficientps Goto Github PK

Traceback (most recent call last):
File "train.py", line 148, in
main()
File "train.py", line 144, in main
meta=meta)
File "/data/PycharmProjects/EfficientPS/mmdet/apis/train.py", line 112, in train_detector
meta=meta)
File "/data/PycharmProjects/EfficientPS/mmdet/apis/train.py", line 245, in _non_dist_train
runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
File "/home/sun/anaconda3/envs/efficientPS_env/lib/python3.7/site-packages/mmcv/runner/runner.py", line 384, in run
epoch_runner(data_loaders[i], **kwargs)
File "/home/sun/anaconda3/envs/efficientPS_env/lib/python3.7/site-packages/mmcv/runner/runner.py", line 283, in train
self.model, data_batch, train_mode=True, **kwargs)
File "/data/PycharmProjects/EfficientPS/mmdet/apis/train.py", line 75, in batch_processor
losses = model(**data)
File "/home/sun/anaconda3/envs/efficientPS_env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/sun/anaconda3/envs/efficientPS_env/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward
return self.module(*inputs[0], **kwargs[0])
File "/home/sun/anaconda3/envs/efficientPS_env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/data/PycharmProjects/EfficientPS/mmdet/core/fp16/decorators.py", line 49, in new_func
return old_func(*args, **kwargs)
File "/data/PycharmProjects/EfficientPS/mmdet/models/efficientps/base.py", line 147, in forward
return self.forward_train(img, img_metas, **kwargs)
File "/data/PycharmProjects/EfficientPS/mmdet/models/efficientps/efficientPS.py", line 205, in forward_train
proposal_list = self.rpn_head.get_bboxes(*proposal_inputs)
File "/data/PycharmProjects/EfficientPS/mmdet/core/fp16/decorators.py", line 127, in new_func
return old_func(*args, **kwargs)
File "/data/PycharmProjects/EfficientPS/mmdet/models/anchor_heads/anchor_head.py", line 276, in get_bboxes
scale_factor, cfg, rescale)
File "/data/PycharmProjects/EfficientPS/mmdet/models/anchor_heads/rpn_head.py", line 92, in get_bboxes_single
proposals, _ = nms(proposals, cfg.nms_thr)
File "/data/PycharmProjects/EfficientPS/mmdet/ops/nms/nms_wrapper.py", line 56, in nms
inds = nms_cuda.nms(dets_th, iou_thr)
RuntimeError: !dets.type().is_cuda() INTERNAL ASSERT FAILED at mmdet/ops/nms/src/nms_cpu.cpp:7, please report a bug to PyTorch. dets must be a CPU tensor (nms_cpu_kernel at mmdet/ops/nms/src/nms_cpu.cpp:7)
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) + 0x6d (0x7f17694182ad in /home/sun/anaconda3/envs/efficientPS_env/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #1: at::Tensor nms_cpu_kernel(at::Tensor const&, float) + 0x86f (0x7f16878d9b6f in /data/PycharmProjects/EfficientPS/mmdet/ops/nms/nms_cpu.cpython-37m-x86_64-linux-gnu.so)
frame #2: nms(at::Tensor const&, float) + 0xea (0x7f16878c43da in /data/PycharmProjects/EfficientPS/mmdet/ops/nms/nms_cpu.cpython-37m-x86_64-linux-gnu.so)
frame #3: + 0x3f3bb (0x7f16878863bb in /data/PycharmProjects/EfficientPS/mmdet/ops/nms/nms_cuda.cpython-37m-x86_64-linux-gnu.so)
frame #4: + 0x3bc94 (0x7f1687882c94 in /data/PycharmProjects/EfficientPS/mmdet/ops/nms/nms_cuda.cpython-37m-x86_64-linux-gnu.so)

No code here.

Hi,

Did you use an existing implementation of this loss or have you created your own? Can you please me point in a direction where I can find this implementation?

Thanks for the repos, i'm currently trying to train my own dataset, is it possible to train my own dataset with this repo? how?

Dear authors,

Thanks for your great work! I wanna try this repo, but the links of the pre-trained models seem to be wrong? I could not download them. I'm not sure if it's my network. Could you send them to my mailbox([email protected])? Thanks a lot!

Thanks for your great work. I download your pre-trained models and configfiles, then I run the tools/cityscapes_save_predictions.py directly. Howerver, I got result like these:

.The command I runned is :
python tools/cityscapes_save_predictions.py configs/efficientPS_cityscapes.py models/cityscapes_model.pth imgs res_i
Are there anything else I haven't done? Can you help me？

Hi,
Thanks for sharing the code with the community.

I would like to know how to calculate the mIoU on the Cityscapes validation set.

I am trying to work towards Panoptic segmentation but getting stuck while preparing the dataset. In the cityscapes websites, there are many downloadables (gtFine_trainvaltest.zip, leftImg8bit_trainvaltest.zip, etc), Which one should I download?

I tried with the "gtFine_trainvaltest.zip" set and while using convert_cityscapes.py, it shows me the following error

FileNotFoundError: [Errno 2] No such file or directory: '~implementations/gtFine_trainvaltest/leftImg8bit/train/bremen/bremen_000045_000019_leftImg8bit.png'

Pre-Trained Models is not available, the file in the corresponding link cannot be downloaded.

Hi, thanks for your great work! But when I try to run the train/test demo, my GPU memory ran out
(on Tesla V100, 16GB), so I just wonder how should I change the hyperparameters in config file to reduce GPU consumption and still get good performance in the mean while ?

Thanks a lot.

hello, thanks for your work.Could you please tell me how to convert the model to onnx.I have tried to use the code of mmdetection, but failed.

Hi
Thanks for the great work and sharing code. I tried to use " python tools/convert_cityscapes.py" but getting error "ModuleNotFoundError: No module named 'pycococreatortools'". I installed python api of coco dataset and still getting this error. Any Idea why is this happening?

kind regards

Hi there!
Thanks for sharing your code! :)
I tried to run the code, but I don't have access to a Titan RTX as you proposed in the paper... Obviously I'm running out of GPU memory... :)
I read the paper also, really nice work!
Do you think there is a way to use your proposed procedure on "smaller", more widely used GPUs, or downscale the networks to process smaller images?
Thanks!

Dear DeepSceneSeg Team,

in the output there are some pixel labeled as unknown class (for example in output of KITTI).

Why?

if it was wanted then why didn't you choose to label the semantic head in panoptic output when the class is labeled as unknown?

Thanks for your attention!

Dear author,

Thanks for sharing the model
I tried to download the pretrained weights, both links are not working for me..
it says 502 Bad Gateway

Could you please recheck the links? Thank :)

Thanks for your great work authors. Is there a plan to release check point for IDD?

Dear authors,

Thanks for your great work! I Would like to use the output of your model for my research. Therefore, the pre-trained models on the mapillary (and cityscapes) dataset would be of great help. Can you guys either upload the model on github or send them to me privately? Would highly appreciate it!

Thanks in advance and looking forward to your answer.

Cheers, Seger

Do I need to run createPanopticImgs.py as preparation for cityscapes for KITTI dataset? To prepare the data to coco panoptic segmentation format?

Hello ,thanks for sharing the pre-trained model. I want to ask if the pre-trained model (cityscape) detect other classes except classes like rider and car? I have run your code and the model can detect cars and people successfully, but other classes like road, vegetation are missing. As you provided results of other classes in cityscape-dataset, I asume the pre-trained model should be able to predict other classes. Could you explain a bit more. Thanks! :)

I tried to download your Pre-Trained Model for Cityscapes but it doesn't work. Can you double check if the link is still valid please? Thanks.

Hi, nice work! where is the script to train on the kitti subset you provide?

HI, thanks for the great repo.

I'm doing some inference with the script cityscapes_demo.py
It can generate good result, but only semantic seg result.

I changed the param eval='panoptic' or 'semantic' or 'instance' or 'segment':
result = inference_detector(model, img_, eval='panoptic')
the result always the same (semantic segmentation result) like this:

So the param eval can't work?
I want to know the reason and the other question:
can the demo scripts be used as inference tools for other pictures?

Thanks

Thanks for this great work!

Is there any way to let us access the KITTI panoptic dataset provided in the website?

The original link is broken.

line 365, mmdet/models/efficientps/efficientPS.py
Directly resize the semantic logits to ref_size may cause dislocation.
It should be resized to pad_shape and cropped to ref_size.
This bug occurs when I use another dataset. I'm not sure whether it occurs in the original dataset.

使用./tools/dist_test.sh efficientPS_multigpu_sample.py ${CHECKPOINT_FILE} ${GPU_NUM} --eval panoptic推理时报错

Dear authors,

Thanks again for your great contribution. I want to use the output of EfficientPS for further processing in my own research. Therefore, my main interest is running inference using your pretrained mapillary vista's (or temporarily Cityscapes) model. However, I get an error when running the test-file with multiple GPU's with nGPUs set to 1. A transcript of the record is provided at the bottom. I tried updating PyTorch to 1.6, but no luck either. Do you have a clue what it could be?
I recon it could have to do something with the provided pre-trained model, as training does seem to work..

Additional information:

• I set up the environment exactly as recommended, using the environment.yml and requirements.txt files.
• When installing efficientPS implementation, I also get a lot of depracated warnings. However, it does finalize the installation.
• Training does seem to work as it should

My settings:

• CUDA 10.2.89
• GCC 8.4.0
• Python 3.7

Running python -c 'from torch.utils.collect_env import main; main() gives the following:

(efficientPS_env) segetak@cmtcde5u157:~/thesis/EfficientPS$ python -c 'from torch.utils.collect_env import main; main()'
Collecting environment information...
PyTorch version: 1.4.0
Is debug build: No
CUDA used to build PyTorch: 10.1

OS: Ubuntu 18.04.5 LTS
GCC version: (Ubuntu 8.4.0-1ubuntu1~18.04) 8.4.0
CMake version: version 3.10.2

Python version: 3.7
Is CUDA available: Yes
CUDA runtime version: 10.2.89
GPU models and configuration: GPU 0: GeForce GTX 980
Nvidia driver version: 460.32.03
cuDNN version: Could not collect

Versions of relevant libraries:
[pip3] numpy==1.20.2
[pip3] torch==1.4.0
[pip3] torchvision==0.5.0
[conda] torch                     1.4.0                    pypi_0    pypi
[conda] torchvision               0.5.0                    pypi_0    pypi

And finally, the error itself:

(efficientPS_env) segetak@cmtcde5u157:~/thesis/EfficientPS$ ./tools/dist_test.sh pre_trained_models/efficientPS_cityscapes/config/efficientPS_multigpu_sample.py pre_trained_models/efficientPS_cityscapes/model/model.pth 1 --eval panoptic
loading annotations into memory...
Done (t=0.17s)
creating index...
index created!
Traceback (most recent call last):
  File "./tools/test.py", line 169, in <module>
    main()
  File "./tools/test.py", line 135, in main
    checkpoint = load_checkpoint(model, args.checkpoint, map_location='cpu')
  File "/lhome/segetak/anaconda3/envs/efficientPS_env/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 218, in load_checkpoint
    checkpoint = _load_checkpoint(filename, map_location)
  File "/lhome/segetak/anaconda3/envs/efficientPS_env/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 194, in _load_checkpoint
    checkpoint = torch.load(filename, map_location=map_location)
  File "/lhome/segetak/anaconda3/envs/efficientPS_env/lib/python3.7/site-packages/torch/serialization.py", line 527, in load
    with _open_zipfile_reader(f) as opened_zipfile:
  File "/lhome/segetak/anaconda3/envs/efficientPS_env/lib/python3.7/site-packages/torch/serialization.py", line 224, in __init__
    super(_open_zipfile_reader, self).__init__(torch._C.PyTorchFileReader(name_or_buffer))
RuntimeError: version_ <= kMaxSupportedFileFormatVersion INTERNAL ASSERT FAILED at /pytorch/caffe2/serialize/inline_container.cc:132, please report a bug to PyTorch. Attempted to read a PyTorch file with version 3, but the maximum supported version for reading is 2. Your PyTorch installation may be too old. (init at /pytorch/caffe2/serialize/inline_container.cc:132)
frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x33 (0x7f4277e73193 in /lhome/segetak/anaconda3/envs/efficientPS_env/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #1: caffe2::serialize::PyTorchStreamReader::init() + 0x1f5b (0x7f4217fd79eb in /lhome/segetak/anaconda3/envs/efficientPS_env/lib/python3.7/site-packages/torch/lib/libtorch.so)
frame #2: caffe2::serialize::PyTorchStreamReader::PyTorchStreamReader(std::string const&) + 0x64 (0x7f4217fd8c04 in /lhome/segetak/anaconda3/envs/efficientPS_env/lib/python3.7/site-packages/torch/lib/libtorch.so)
frame #3: <unknown function> + 0x6c6536 (0x7f427ce5c536 in /lhome/segetak/anaconda3/envs/efficientPS_env/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #4: <unknown function> + 0x295a74 (0x7f427ca2ba74 in /lhome/segetak/anaconda3/envs/efficientPS_env/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #5: _PyMethodDef_RawFastCallDict + 0x24d (0x55e2e15d044d in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #6: _PyCFunction_FastCallDict + 0x21 (0x55e2e15d05d1 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #7: _PyObject_Call_Prepend + 0x63 (0x55e2e15ce4d3 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #8: PyObject_Call + 0x6e (0x55e2e15c0ffe in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #9: <unknown function> + 0x9e62d (0x55e2e153862d in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #10: _PyObject_FastCallKeywords + 0x128 (0x55e2e1606968 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #11: _PyEval_EvalFrameDefault + 0x52fe (0x55e2e166c39e in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #12: _PyEval_EvalCodeWithName + 0x5da (0x55e2e15aeb0a in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #13: _PyFunction_FastCallDict + 0x1d5 (0x55e2e15af925 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #14: _PyObject_Call_Prepend + 0x63 (0x55e2e15ce4d3 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #15: <unknown function> + 0x16bd5a (0x55e2e1605d5a in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #16: _PyObject_FastCallKeywords + 0x128 (0x55e2e1606968 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #17: _PyEval_EvalFrameDefault + 0x49e6 (0x55e2e166ba86 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #18: _PyEval_EvalCodeWithName + 0x2f9 (0x55e2e15ae829 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #19: _PyFunction_FastCallKeywords + 0x387 (0x55e2e15ff107 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #20: _PyEval_EvalFrameDefault + 0x14e5 (0x55e2e1668585 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #21: _PyEval_EvalCodeWithName + 0x2f9 (0x55e2e15ae829 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #22: _PyFunction_FastCallKeywords + 0x325 (0x55e2e15ff0a5 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #23: _PyEval_EvalFrameDefault + 0x416 (0x55e2e16674b6 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #24: _PyEval_EvalCodeWithName + 0x2f9 (0x55e2e15ae829 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #25: _PyFunction_FastCallKeywords + 0x387 (0x55e2e15ff107 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #26: _PyEval_EvalFrameDefault + 0x14e5 (0x55e2e1668585 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #27: _PyFunction_FastCallKeywords + 0xfb (0x55e2e15fee7b in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #28: _PyEval_EvalFrameDefault + 0x416 (0x55e2e16674b6 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #29: _PyEval_EvalCodeWithName + 0x2f9 (0x55e2e15ae829 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #30: PyEval_EvalCodeEx + 0x44 (0x55e2e15af714 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #31: PyEval_EvalCode + 0x1c (0x55e2e15af73c in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #32: <unknown function> + 0x22cf14 (0x55e2e16c6f14 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #33: PyRun_FileExFlags + 0xa1 (0x55e2e16d1331 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #34: PyRun_SimpleFileExFlags + 0x1c3 (0x55e2e16d1523 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #35: <unknown function> + 0x238655 (0x55e2e16d2655 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #36: _Py_UnixMain + 0x3c (0x55e2e16d277c in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)
frame #37: __libc_start_main + 0xe7 (0x7f4281a36bf7 in /lib/x86_64-linux-gnu/libc.so.6)
frame #38: <unknown function> + 0x1dcff0 (0x55e2e1676ff0 in /lhome/segetak/anaconda3/envs/efficientPS_env/bin/python)

Traceback (most recent call last):
  File "/lhome/segetak/anaconda3/envs/efficientPS_env/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/lhome/segetak/anaconda3/envs/efficientPS_env/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/lhome/segetak/anaconda3/envs/efficientPS_env/lib/python3.7/site-packages/torch/distributed/launch.py", line 263, in <module>
    main()
  File "/lhome/segetak/anaconda3/envs/efficientPS_env/lib/python3.7/site-packages/torch/distributed/launch.py", line 259, in main
    cmd=cmd)
subprocess.CalledProcessError: Command '['/lhome/segetak/anaconda3/envs/efficientPS_env/bin/python', '-u', './tools/test.py', '--local_rank=0', 'pre_trained_models/efficientPS_cityscapes/config/efficientPS_multigpu_sample.py', 'pre_trained_models/efficientPS_cityscapes/model/model.pth', '--launcher', 'pytorch', '--eval', 'panoptic']' returned non-zero exit status 1.

Hello authors. Thank you for the great work.
In your paper you reported that the model achieves mIoU of 84.2% and AP of 39.1% but I can't figure out how to output mIoU and mAP. I tried passing --eval panoptic cityscapes and other metrics when calling test.py but not thing work so far. Below is the traceback.
Edit: Actually I have tried calculate mIoU from the model output and it is quite close to your result. But I can not find a way to get the model confidence score of the bounding box for AP calculation. How should I do it?

Traceback (most recent call last):
  File "/content/EfficientPS/tools/test.py", line 171, in <module>
    main()
  File "/content/EfficientPS/tools/test.py", line 167, in main
    dataset.evaluate(outputs, args.eval, **kwargs)
  File "/content/EfficientPS/mmdet/datasets/cityscapes.py", line 244, in evaluate
    classwise, proposal_nums, iou_thrs))
  File "/content/EfficientPS/mmdet/datasets/coco.py", line 327, in evaluate
    result_files, tmp_dir = self.format_results(results, jsonfile_prefix)
  File "/content/EfficientPS/mmdet/datasets/coco.py", line 281, in format_results
    format(len(results), len(self)))
AssertionError: The length of results is not equal to the dataset len: 0 != 500

hi! The design of the network is very clever and it has achieved good results! But what I don't understand is why intermediate logits in fusion modeul are needed. According to my understanding，argmax(intermediate logits) actually represents that the possible instance part（In addition to the fusion results） of the semantic logits are accepted.
Hope to get your reply！
good luck.

Hello,

I want to do inference with EfficientPS on KITTI360 images. Therefore i just wanted to try to use the network with the pretrained weights trained on Cityscapes. Because KITTI-360 classes are compatible with the Cityscapes Dataset. My Problem is that the command below fails. I use the command but the pathes are shortened. I changed the cityscapes_inference.py so that the one argument is not config but -config.
Command:
python -m torch.distributed.launch --nproc_per_node=1 tools/cityscapes_inference.py -config="/EfficientPS/efficientPS_cityscapes/config/efficientPS_multigpu_sample.py" checkpoint="/EfficientPS/efficientPS_cityscapes/model/model.pth" input="/KITTI-360/data_2d_raw/2013_05_28_drive_0000_sync/image_00/data_rect" out="/KITTI-360/InferenceInstance"
Error:
OSError: checkpoint=/EfficientPS/efficientPS_cityscapes/model/model.pth is not a checkpoint file

I downloaded the EfficientPS pre-trained weights and unziped them in the EfficientPS folder. I thought model.pth is a checkpoint file with pre-trained weights.
I would be thankful if anyone could tell me how to do inference correctly with the pre-trained weights.

Thank you in advance,
Marcus

I trained the model from scratch using Cityscapes dataset and used it to run inference. My gpu is Quadro RTX 6000 96 GB.
I got 0.61 fps for inference time or 1637 ms for 1 image when it should takes 166 ms for 1 image according to the paper. Anyone got the same problem or know what could potentially cause my inference time too slow? Thanks.

Hi
This is an output of EfficientPS without blend of original image.

Why are there shadows close to cars and other dynamic elements in the image? is it an error?

Thank you

Hi,
Question:
In the code of cityscapes_save_predictions.py this part in the code is the panoptic segmentation with the semantic segmentation encode ?

pan_pred, cat_pred, _ = result[0]
imageId = imgName.replace("_leftImg8bit.png", "")
inputFileName = imgName
outputFileName = imgName.replace("_leftImg8bit.png", "_panoptic.png")
img = Image.open(os.path.join(path, imgName))
out_path = os.path.join(out_dir, outputFileName)
sem = cat_pred[pan_pred].numpy()
sem_tmp = sem.copy()
sem_tmp[sem==255] = colors.shape[0] - 1
sem_img = Image.fromarray(colors[sem_tmp])
is_background = (sem < 11) | (sem == 255)
pan_pred = pan_pred.numpy() 
pan_pred[is_background] = 0

for example: in the output of the cityscapes_save_predictions.py there are cars with the same color without distinct color for each instance.

Thanks for watching!

Thanks for providing the code. However I am currently training EfficientPS using the Cityscapes Dataset. I am a bit curious about the MultiStep Learning Rate Scheduler explained in the Paper.

In the paper it is said that a linear warmup from 1/3 * base to base is performed, than 10 epochs with 1e-4 and than the base learning rate decreased by a factor of 10 at every milestone. However I can not find the implementation for the 10 epochs in the code and also the freezing of the iABN layers.

How is your Cityscapes model trained, that you provide?

(venv) root@cc3180440f4b:/opendr/src/opendr/perception/panoptic_segmentation/efficient_ps/algorithm/EfficientPS# python tools/convert_cityscapes.py ./data_2/ ./data/cityscapes/
Loading Cityscapes from ./data_2/
Converting train ...
0%| | 0/2974 [00:00<?, ?it/s]
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/usr/lib/python3.8/multiprocessing/pool.py", line 48, in mapstar
return list(map(*args))
File "tools/convert_cityscapes.py", line 163, in call
coco_ann_i = pct.create_annotation_info(
File "/opendr/venv/lib/python3.8/site-packages/pycococreatortools/pycococreatortools.py", line 99, in create_annotation_info
segmentation = binary_mask_to_polygon(binary_mask, tolerance)
File "/opendr/venv/lib/python3.8/site-packages/pycococreatortools/pycococreatortools.py", line 48, in binary_mask_to_polygon
contours = np.subtract(contours, 1)
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "tools/convert_cityscapes.py", line 198, in
main(parser.parse_args())
File "tools/convert_cityscapes.py", line 76, in main
for coco_img, coco_ann in tqdm.tqdm(pool.imap(worker, img_list, 8), total=total):
File "/opendr/venv/lib/python3.8/site-packages/tqdm/std.py", line 1166, in iter
for obj in iterable:
File "/usr/lib/python3.8/multiprocessing/pool.py", line 420, in
return (item for chunk in result for item in chunk)
File "/usr/lib/python3.8/multiprocessing/pool.py", line 868, in next
raise value
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
(venv) root@cc3180440f4b:/opendr/src/opendr/perception/panoptic_segmentation/efficient_ps/algorithm/EfficientPS#

How can I solve the above error

I have installed pycococreator and pycocotools from https://github.com/waspinator/pycococreator

I flowed the instructions to create a conda environment. When i tried the test script i got the error:

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject

But could solve it updating the mmcv from 0.5.9 to 0.6.0 .

Then, when i ran the script i got this error:

Traceback (most recent call last):
  File "tools/test.py", line 169, in <module>
    main()
  File "tools/test.py", line 135, in main
    checkpoint = load_checkpoint(model, args.checkpoint, map_location='cpu')
  File "/opt/miniconda3/envs/env/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 224, in load_checkpoint
    checkpoint = _load_checkpoint(filename, map_location)
  File "/opt/miniconda3/envs/env/lib/python3.7/site-packages/mmcv/runner/checkpoint.py", line 200, in _load_checkpoint
    checkpoint = torch.load(filename, map_location=map_location)
  File "/opt/miniconda3/envs/env/lib/python3.7/site-packages/torch/serialization.py", line 587, in load
    with _open_zipfile_reader(opened_file) as opened_zipfile:
  File "/opt/miniconda3/envs/env/lib/python3.7/site-packages/torch/serialization.py", line 242, in __init__
    super(_open_zipfile_reader, self).__init__(torch._C.PyTorchFileReader(name_or_buffer))
RuntimeError: [enforce fail at inline_container.cc:222] . file not found: efficientPS_cityscapes/version

Seems like i'm missing something in the weights zip file :/

The command i used:

(env) Singularity> python tools/test.py configs/efficientPS_singlegpu_sample.py models/efficientPS_cityscapes.zip --eval panoptic

HI, thanks for the great repo.

I am doing simple testing with ur script cityscapes_inference.py
it worked well with cityspace images while the size is 1024*2048
but for othersize images,it is somehow not correct.

it has a doubled results with the instance part and the semantic head results not overlapped with each other..
so ended up with two results separate.

the problems lies in there where the mask_pred prediction positions are wrong (not to ref size)

            bbx_inv = invert_roi_bbx(det_bboxes[:, :4], 
                      tuple(mask_pred.shape[2:]), ref_size)
            bbx_idx = torch.arange(0, det_bboxes.size(0), 
                      dtype=torch.long, device=det_bboxes.device)
            
            mask_pred = roi_sampling(mask_pred, bbx_inv, bbx_idx, 
                        ref_size, padding="zero")

am a bit confused by the script, looks like an easy bug, but havent found a proper way to solve it. could you please take a look? Thanks

I get a nearly random segmentation with kitti pre-trained model.
Maybe the kitti pre-trained model was wrongly uploaded? I can get a not too bad kitti segmentation result with Cityscapes pre-trained model. But, some cars are misdetected with this pre-trained model, which may caused by domain gap between Cityscapes and Kitti.

Could you please re-upload the kitti pre-trained model? I will appreciate for your response.

hello, thanks for your great work.I have prepared Mapillary dataset,but there's something wrong about configs and lr schedule.
CUDA_ERROR_ILLEGAL_ADDRESS: an illegal memory access was encountered happened,I think it may be caused by configs.
If it's possible to provide your mapillary training config file?

Dear authors,

What exactly does "hasInstance" do? When I experiment with putting different class labels on True/False, I get very different results. Can you elaborate a bit on how "hasInstances" is usually used, and when it should be set to False or True?

I though the following:
False = The label can only have one single instance
True = The label can have multiple instances
But when I set it to False, it seems like the model completely forgets about the label... Is this nomal?

Thanks in advance and kind regards

hi, I am not find efficientPS_singlegpu_sample.py file

Hello. It's been very long. Please upload your code and pretrained model

Hi,

I am currently attempting to install your code, however I am unable to get the requirement inplace_abn installed. I'm running Ubuntu 18.04.
I followed your instructions: I created a new conda environment using your environment.yaml file and also installed

• cudatoolkit=10.2
• pytorch=1.7.0
• torchvision=0.8.0
  I also had to uninstall "typing" from pip to progress past an error (https://stackoverflow.com/questions/55833509/attributeerror-type-object-callable-has-no-attribute-abc-registry). With typing installed, a get an error: "AttributeError: type object 'Callable' has no attribute '_abc_registry'

My current error is with the dependency inplace_abn, which from my reading of their issues list lacks support for CUDA Driver Version 11. I'm currently using CUDA Driver Version 11.4. My error is a compilation error attempting to install inplace_abn (I'm reluctant to change my CUDA driver version due to other dependencies in my project).

My question is, will your network run without inplace_abn at inference time only, with code modifications? (or will the different BN scaling parameters ruin the network performance if inplace_abn is removed and replaced with traditional BN)

Dear authors,

Thank you very much for this fine piece of science. I am using your work as inspiration for my own research. While I was using your model, I came across a memory leak issue in mean_ap. On line 310, a multiprocessing pool is opened. However, this one is never closed, leading to memory leak and oom issues on my system (I use a 32gb video ram, so I was surprised). The pool should be closed after line 365, this is also an issue known to the authors of mmdetection, and solved in a more recent version of their code.

Also, I have a question about the num_classes parameter in the model config file for training cityscapes. For the semantic_head part, this is set to 19. I can understand this, as the total number of the stuff + thing classes for cityscapes is 19, as is defined by the trainingId in the cityscapesscripts helpers/labels.py file. However, for the bbox_head and the mask_head, this is set to 9. Is this value a result of only the thing classes (8) + 1 (background)= 9? Does that mean that the bbox_head and the mask_head can only predict 8 classes (the thing classes), while there are 19 classes in total? I am confused by these values, because in my opinion you also want your bbox_head and mask_head to classify stuff_classes. Can you elaborate on how you came to these values?

Thanks in advance for answering my question.

Hi,
I want obtain different color for different intances of the same class, but if I put an input image I have this result:

If I use a combined code with "tools/cityscapes_inference.py" and "tools/cityscapes_save_predictions.py" I have istances with the same color and their boundaries white.

Is this the final output of the network? or did I skip a step?

If I skipped a step, which step and code can I use to obtain the final panoptic output result?