shyam671 / mask2anomaly-unmasking-anomalies-in-road-scene-segmentation Goto Github PK

Dear authors,

Congratulations on this very interesting work! I would like to reproduce the experiments in table 1 of the paper as a baseline for our work, however, after following the code in this repo to train M2A on Cityscapes and then finetune with COCO images I get different results. On the datasets evaluated in Table 1.

For instance, I get:

Lost and Found -> AUPRC: 51.95 FPR@95: 35.22
RoadAnomaly -> AUPRC: 75.47 FPR@95: 18.47
RoadObstacle21 -> AUPRC: 28.48 FPR@95: 1.19
RoadAnomaly21 -> AUPRC: 87.33 FPR@95: 17.49
Fishyscapes -> AUPRC: 80.02 FPR@95: 28.56

From the code I get that there is no fixed seed during training and finetuning? Is that correct? If so, did you ever experiment with different runs? Perhaps there is some "trick" that should be done during evaluation that I missed? I just ran anomaly_utils/anomaly_inference.py as specified in the README. I also downloaded the datasets from the link provided to make sure they were the same.

Thanks again for your help.
Pau

I try to downlowd this picture to test the inference (!wget https://www.epfl.ch/labs/cvlab/wp-content/uploads/2019/10/road_anomaly_gt_contour.jpg ), and i change its name.

And then i run this commend (CUDA_VISIBLE_DEVICES=1 python anomaly_utils/anomaly_inference.py --input '/home/znx/Mask2Anomaly-Unmasking-Anomalies-in-Road-Scene-Segmentation/*.jpg' --config-file '/home/znx/Mask2Anomaly-Unmasking-Anomalies-in-Road-Scene-Segmentation/configs/cityscapes/semantic-segmentation/anomaly_inference.yaml' ).

The error occurs.(> to CPU due to CUDA OOM
[09/27 16:57:46 detectron2]: /home/znx/src/Mask2Anomaly/1.jpg: finished in 13.62s
Traceback (most recent call last):
File "anomaly_utils/anomaly_inference.py", line 225, in
ood_out = anomaly_scores[ood_mask]
IndexError: too many indices for array: array is 4-dimensional, but 5 were indexed
)

How can i solve it？

Dear authors,
I run this commend（CUDA_VISIBLE_DEVICES=0 python train_net.py --num-gpus 1 --config-file configs/cityscapes/semantic-segmentation/anomaly_train.yaml）, the following issues will occur：

eta: 1 day, 7:55:10 iter: 4999 total_loss: 43.6 loss_ce: 0.873 loss_mask: 1.135 loss_dice: 2.291 loss_ce_0: 1.078 loss_mask_0: 1.183 loss_dice_0: 2.487 loss_ce_1: 0.951 loss_mask_1: 1.184 loss_dice_1: 2.416 loss_ce_2: 0.8681 loss_mask_2: 1.126 loss_dice_2: 2.373 loss_ce_3: 0.8447 loss_mask_3: 1.139 loss_dice_3: 2.31 loss_ce_4: 0.9072 loss_mask_4: 1.116 loss_dice_4: 2.314 loss_ce_5: 0.8622 loss_mask_5: 1.144 loss_dice_5: 2.321 loss_ce_6: 0.9052 loss_mask_6: 1.136 loss_dice_6: 2.297 loss_ce_7: 0.8597 loss_mask_7: 1.153 loss_dice_7: 2.337 loss_ce_8: 0.9218 loss_mask_8: 1.145 loss_dice_8: 2.318 time: 1.3567 last_time: 1.2917 data_time: 0.4710 last_data_time: 0.3988 lr: 9.4987e-05 max_mem: 16472M

Traceback (most recent call last):
File "train_net.py", line 321, in
launch(
File "/usr/local/lib/python3.8/dist-packages/detectron2/engine/launch.py", line 84, in launch
main_func(*args)
File "train_net.py", line 315, in main
return trainer.train()
File "/usr/local/lib/python3.8/dist-packages/detectron2/engine/defaults.py", line 486, in train
super().train(self.start_iter, self.max_iter)
File "/usr/local/lib/python3.8/dist-packages/detectron2/engine/train_loop.py", line 156, in train
self.after_step()
File "/usr/local/lib/python3.8/dist-packages/detectron2/engine/train_loop.py", line 190, in after_step
h.after_step()
File "/usr/local/lib/python3.8/dist-packages/detectron2/engine/hooks.py", line 556, in after_step
self._do_eval()
File "/usr/local/lib/python3.8/dist-packages/detectron2/engine/hooks.py", line 529, in _do_eval
results = self._func()
File "/usr/local/lib/python3.8/dist-packages/detectron2/engine/defaults.py", line 455, in test_and_save_results
self._last_eval_results = self.test(self.cfg, self.model)
File "/usr/local/lib/python3.8/dist-packages/detectron2/engine/defaults.py", line 619, in test
results_i = inference_on_dataset(model, data_loader, evaluator)
File "/usr/local/lib/python3.8/dist-packages/detectron2/evaluation/evaluator.py", line 213, in inference_on_dataset
results = evaluator.evaluate()
File "/usr/local/lib/python3.8/dist-packages/detectron2/evaluation/cityscapes_evaluation.py", line 186, in evaluate
results = cityscapes_eval.evaluateImgLists(
File "/dfs/data/cityscapesScripts/cityscapesscripts/evaluation/evalPixelLevelSemanticLabeling.py", line 475, in evaluateImgLists
nbPixels += evaluatePair(predictionImgFileName, groundTruthImgFileName, confMatrix, instStats, perImageStats, args)
File "/dfs/data/cityscapesScripts/cityscapesscripts/evaluation/evalPixelLevelSemanticLabeling.py", line 599, in evaluatePair
confMatrix[gt_id][pred_id] += c
IndexError: index 255 is out of bounds for axis 0 with size 34

How can i solve it？ Thank you！

Hi, I'm appreciate you to share this great work. However, during process, a problem occur with the model prediction results seems to have label '255', which leads to error reported when we need to calculate confusion matrix(34 * 34).

My question is in the cityscapes dataset pre-process stage, should I need to do any other work except the following command you mentioned in README:
CITYSCAPES_DATASET=/path/to/abovementioned/cityscapes python cityscapesscripts/preparation/createTrainIdLabelImgs.py

or this error occurs due to other issues? hope that you can share your opnions, very thanks!

Dear authors, it would be very convenient if you add a LICENSE to your repository. This would allow third parties to build on your code and release it in a modified way. Note that without a LICENSE, it should not be possible to use the software as "open source". See https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/licensing-a-repository

Hello, validation dataset removed today , I need it ! it had been available before 4 hours ago.
could you mind make it available ? ?
thanks

I have noticed that in all your configuration files you are using INPUT.FORMAT: "RGB" while, during inference, the image is read using img = read_image(path, format="BGR") .
Since I was assuming that the network is trained on RGB images it would benefit to maintain that format. Is there a format conversion (from RGB to BGR) before using the samples as training input? Or is there a reason to use two different format for training/test purposes?

Hi, I followed the README to run the code. But the stage2 loss didn't decrease, and the result is very strange.what's the problem?

Hello，
Congratulations on a great job！I have a question. For contrast loss, both normal and abnormal l_N are negative. The paper says that under ideal circumstances, normal l_N is -1 and abnormal is 0. Why does the formula for contrast loss seem to be to make normal l_N as close to 0 as possible and abnormal as much as possible greater than m(that is, 0.75), that is, abnormal is also as close to 0 as possible? Thank you very much!

Dear authors,
congratulations for your great work!

I was using your method in inference mode and I realized that the visualization of the anomaly map values sometimes contains some red spots that can be treated as anomaly:

By analyzing them, we can understand that they are negative values resulting from anomaly_map = 1 - torch.max(anomaly_map[0:19,:,:], axis = 1)[0]. Probably the cv2.applyColorMap is not able to correctly map negative values. Thus, to improve the visualization (and deleting red spots), does it make sense to normalize those anomaly score values between 0 and 1 before using cv2.applyColorMap? See the image result:

Additional question: why, in the first place, do we have > 1 values in outputs["sem_seg"] where outputs = predictor(im)? I am assuming that the values should be between [0, 1] since they are the output of $softmax(C)^T \times sigmoid(M)$ and anomaly_map = 1 - torch.max(anomaly_map[0:19,:,:], axis = 1)[0] should represents the $f(x)$ in equation (2) of the paper.

Thank you so much in advance for your answers!

Hello, I want to know why you add an if condition here?