pingolh / fchardnet

Fully Convolutional HarDNet for Segmentation in Pytorch

FCHarDNet

Aug-15-2021 Update

• Added a new v2_transform() method to replace torch.cat + nn.Conv2d combinations with CatConv2d, an all-in-1 fused cuda kernel combining Concat and Conv2d.

• Inference on TitanV speedup from 70 fps to 99 fps

• CatConv2d Installation

cd CatConv2d/
python setup.py install

(Please note that backward path for CatConv2d hasn't been implemented)

Fully Convolutional HarDNet for Segmentation in Pytorch

• Implementaion based on Harmonic DenseNet: A low memory traffic network (ICCV 2019)
• Refer to Pytorch-HarDNet for more information about the backbone model
• This repo was forked from meetshah1995/pytorch-semseg

Architecture

• Simple U-shaped encoder-decoder structure
• Conv3x3/Conv1x1 only (including the first layer)
• No self-attention layer or Pyramid Pooling

Results

Method	#Param (M)	GMACs / GFLOPs	Cityscapes mIoU	fps on Titan-V @1024x2048	fps on 1080ti @1024x2048
ICNet	7.7	30.7	69.5	63	48
SwiftNetRN-18	11.8	104	75.5	-	39.9
BiSeNet (1024x2048)	13.4	119	77.7	36	27
BiSeNet (768x1536)	13.4	66.8	74.7	72**	54**
FC-HarDNet-70	4.1	35.4	76.0	70	53
FC-HarDNet-70 V2 (with CatConv2d)	4.1	35.4	76.0	99	63

• ** Speed tested in 1536x768 instead of full resolution.

---

DataLoaders implemented

• CamVid
• Pascal VOC
• ADE20K
• MIT Scene Parsing Benchmark
• Cityscapes
• NYUDv2
• Sun-RGBD

Requirements

• pytorch >=0.4.0
• torchvision ==0.2.0
• scipy
• tqdm
• tensorboardX

Usage

Setup config file

Please see the usage section in meetshah1995/pytorch-semseg

To train the model :

python train.py [-h] [--config [CONFIG]]

--config                Configuration file to use (default: hardnet.yml)

To validate the model :

usage: validate.py [-h] [--config [CONFIG]] [--model_path [MODEL_PATH]] [--save_image]
                       [--eval_flip] [--measure_time]

  --config              Config file to be used
  --model_path          Path to the saved model
  --eval_flip           Enable evaluation with flipped image | False by default
  --measure_time        Enable evaluation with time (fps) measurement | True by default
  --save_image          Enable writing result images to out_rgb (pred label blended images) and out_predID

Pretrained Weights

• Cityscapes pretrained weights: Download
  (Val mIoU: 77.7, Test mIoU: 75.9)
• Cityscapes pretrained with color jitter augmentation: Download
  (Val mIoU: 77.4, Test mIoU: 76.0)
• HarDNet-Petite weights pretrained by ImageNet:
  included in weights/hardnet_petite_base.pth

Prediction Samples

fchardnet's Issues

codes for gflops / gmacs and cio

Hi,
-Could you please share the codes for getting gflops / gmacs and CIO? or what kind of function have you used for calculating macs? (torch == 0.4.1)
-To achieve 35fps on cityscapes dataset, where did you put on start&end point? the time measurement what you calculated in validate.py performs 65 fps on cityscapes with 1024x2048. It's quite reasonable because the GPU time depends on the machine, but if you have, could you please share or mention this code or point as well?

Thank you !

what is the top-1 acc for weights/hardnet_petite_base.pth on imagenet

what is the top-1 acc for weights/hardnet_petite_base.pth on imagenet

about IoU with different size from cityscape,please help me!

Because of the camera output size is 640*360, so I changed the size of cityscapes dataset. Then I use the project code to train, but I can't get the good IoU about 75%, please help me , how can get the good IoU result.
the train log is:
INFO:ptsemseg:Iter [90000/90000] Loss: 0.9804 Time/Image: 0.0165 lr=0.090953
11it [00:05, 2.08it/s]
INFO:ptsemseg:Iter 90000 Val Loss: 1.1422
Overall Acc: 0.902127139034985
INFO:ptsemseg:Overall Acc: : 0.902127139034985
Mean Acc : 0.6061411175875274
INFO:ptsemseg:Mean Acc : : 0.6061411175875274
FreqW Acc : 0.8350029456995808
INFO:ptsemseg:FreqW Acc : : 0.8350029456995808
Mean IoU : 0.48783643289422235
INFO:ptsemseg:Mean IoU : : 0.48783643289422235
INFO:ptsemseg:0: 0.949375256023792
INFO:ptsemseg:1: 0.6434965931893878
INFO:ptsemseg:2: 0.8355899486279862
INFO:ptsemseg:3: 0.36742551411680824
INFO:ptsemseg:4: 0.24889070738206942
INFO:ptsemseg:5: 0.31642120260993717
INFO:ptsemseg:6: 0.28094440896774303
INFO:ptsemseg:7: 0.42495380920802917
INFO:ptsemseg:8: 0.8470327817169933
INFO:ptsemseg:9: 0.4534516073428247
INFO:ptsemseg:10: 0.8743228190634068
INFO:ptsemseg:11: 0.4587472314322872
INFO:ptsemseg:12: 0.310287655352762
INFO:ptsemseg:13: 0.8298468752303512
INFO:ptsemseg:14: 0.42321140100466925
INFO:ptsemseg:15: 0.39785631228298224
INFO:ptsemseg:16: 0.031675380114154154
INFO:ptsemseg:17: 0.09916433094570747
INFO:ptsemseg:18: 0.4761983903783339

and the hardnet.yml is:
model:
    arch: hardnet
data:
    dataset: cityscapes
    train_split: train
    val_split: val
    img_rows: 360
    img_cols: 640
    path: ../cityscape_transformation/
    sbd_path: ../cityscape_transformation/
training:
    train_iters: 90000
    batch_size: 48
    val_interval: 500
    n_workers: 8
    print_interval: 10
    augmentations:
        hflip: 0.5
        rscale_crop: [360, 360]
    optimizer:
        name: 'sgd'
        lr: 0.1
        weight_decay: 0.0005
        momentum: 0.9
    loss:
        name: 'bootstrapped_cross_entropy'
        min_K: 4096
        loss_th: 0.3
        size_average: True
    lr_schedule: 
        name: 'poly_lr'
        max_iter: 9000000
    resume: None
    finetune: None    
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
and I only modify the code train.py here:
    v_loader = data_loader(
        data_path,
        is_transform=True,
        split=cfg["data"]["val_split"],
        img_size=(360,640),
    )
and the code cityscapes_loader.py here:
    def __init__(
        self,
        root,
        split="train",
        is_transform=False,
        img_size=(360, 640),
        augmentations=None,
        img_norm=True,
        version="cityscapes",
        test_mode=False,
    ):
        """__init__

pretrained model: number of epochs?

What is the number of epochs for your pretrained model?
Are the parameters those listed in your config files and default args?
Thank you.

Pretrained Model for ADE20K dataset

Hi,

Can you also upload the pretrained model for ADE20K dataset?

Thank you.

Errors in test.py

When I use this command to test I got error:

python test.py --model_path weights/hardnet70_cityscapes_model.pkl --dataset cityscapes --size 2048,1024 --input [input/a.jpg] --output [output]

The following error occurred. Please point out what needs to be fixed.

Error:
Traceback (most recent call last):
File "test.py", line 119, in
test(args)
File "test.py", line 36, in test
device, model, loader = init_model(args)
File "test.py", line 16, in init_model
data_loader = get_loader("icboard")
File "C:\Users\students\Desktop\rakuseki\seg\FCHarDNet-master\ptsemseg\loader_init_.py", line 27, in get_loader
}[name]
KeyError: 'icboard'

The weight files and input image files are located in the following paths.
--model_path weights/hardnet70_cityscapes_model.pkl
--input input/a.jpg]

how to increase fps

Hi all,

I trained the model from scratch with one of my dataset. When I perform the inference, the actual call takes around 23ms. Why is it so slow? BisNet runs with 9ms. GPU is a Titan RTX.

Code snippet:

model.eval()
with torch.no_grad():
        t = time.time()
        pr_mask = model.forward(x_tensor)
        print(time.time() - t)
x_tensor.shape is torch.Size([1, 3, 2080, 2464])

Thank you in advance.

test.py results in black image

Hallo,
thanks for the nice effort. I have been trying to test the pretrained models but i am unsuccessful as the result is always a black image.

update: the problem was decoding the segmap

How can I run inference on a sample of images?

Hello, thanks for your work.

I would like to use test.py to run inference on a folder of images without having to download entire Cityscapes dataset. Is it possible to do this?

Currently, when I try to do it, I get the following error:

Traceback (most recent call last):
  File "test.py", line 119, in <module>
    test(args)
  File "test.py", line 36, in test
    device, model, loader = init_model(args)
  File "test.py", line 16, in init_model
    data_loader = get_loader("icboard")
  File "/home/rbohare/FCHarDNet/ptsemseg/loader/__init__.py", line 27, in get_loader
    }[name]
KeyError: 'icboard'

Thanks for your help.

how to design the network for different input size?

my image is only 300*800, i find thw iou is poor and i want to imporve the accuracy for segmentation. What should i do to realize it?

Could you share the config details of evaluation ?

Hi, thanks for sharing the nice work.

Could you share the config details of evaluation? I find there are some arguments requiring to set.

To achieve the 76.0% mean IoU on the Cityscapes, did you use the filpped input?
Meanwhile, how to update bn?

Thanks again. Looking forward to your reply.

Error in test.py

When I tried test.py with my image on google colaboratory, I got an error.The commands and errors are shown below.
Is there any solution?

!python test.py --model_path ./weights/hardnet70_cityscapes_model.pkl --dataset cityscapes --input /content/drive/MyDrive/test.jpg --output /content/results/

Traceback (most recent call last):
File "test.py", line 119, in
test(args)
File "test.py", line 36, in test
device, model, loader = init_model(args)
File "test.py", line 21, in init_model
test_mode=True
File "/content/FCHarDNet/ptsemseg/loader/cityscapes_loader.py", line 82, in init
self.images_base = os.path.join(self.root, "leftImg8bit", self.split)
File "/usr/lib/python3.6/posixpath.py", line 80, in join
a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

I'm Japanese and I'm sorry if my English is strange.

FileNotFoundError 'data/Cityscapes/gtFine/train/stuttgart/stuttgart_000016_000019_leftgtFine_labelIds.png'

I cloned the master version of this repository today.
So I encountered the following error.

RUNDIR: runs/hardnet/cur
Found 2976 train images
Found 500 val images
Parameters: 4119257
Using optimizer SGD (
Parameter Group 0
    dampening: 0
    lr: 0.02
    momentum: 0.9
    nesterov: False
    weight_decay: 0.0005
)
INFO:ptsemseg:Using bootstrapped_cross_entropy with {'min_K': 4096, 'loss_th': 0.3, 'size_average': True} params
Using loss functools.partial(<function bootstrapped_cross_entropy2d at 0x7fbeb15d5730>, min_K=4096, loss_th=0.3, size_average=True)
INFO:ptsemseg:No checkpoint found at 'None'
WARN: resizing labels yielded fewer classes
/usr/local/lib/python3.6/dist-packages/torch/optim/lr_scheduler.py:123: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)
/usr/local/lib/python3.6/dist-packages/torch/nn/_reduction.py:44: UserWarning: size_average and reduce args will be deprecated, please use reduction='none' instead.
  warnings.warn(warning.format(ret))
Iter [10/90000]  Loss: 2.6415  Time/Image: 0.0490  lr=0.019998
INFO:ptsemseg:Iter [10/90000]  Loss: 2.6415  Time/Image: 0.0490  lr=0.019998
Iter [20/90000]  Loss: 2.2272  Time/Image: 0.0451  lr=0.019996
INFO:ptsemseg:Iter [20/90000]  Loss: 2.2272  Time/Image: 0.0451  lr=0.019996
Traceback (most recent call last):
  File "./train.py", line 267, in <module>
    train(cfg, writer, logger)
  File "./train.py", line 138, in train
    for (images, labels, _) in trainloader:
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 363, in __next__
    data = self._next_data()
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 971, in _next_data
    return self._process_data(data)
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 1014, in _process_data
    data.reraise()
  File "/usr/local/lib/python3.6/dist-packages/torch/_utils.py", line 395, in reraise
    raise self.exc_type(msg)
FileNotFoundError: Caught FileNotFoundError in DataLoader worker process 3.
Original Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/worker.py", line 185, in _worker_loop
    data = fetcher.fetch(index)
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/content/drive/My Drive/Colab Notebooks/20200831_FCHarDNet/FCHarDNet/ptsemseg/loader/cityscapes_loader.py", line 164, in __getitem__
    lbl = Image.open(lbl_path)
  File "/usr/local/lib/python3.6/dist-packages/PIL/Image.py", line 2809, in open
    fp = builtins.open(filename, "rb")
FileNotFoundError: [Errno 2] No such file or directory: './../data/Cityscapes/gtFine/train/stuttgart/stuttgart_000016_000019_leftgtFine_labelIds.png'

It was solved by the following provisional measures.

[@cityscapes_loader.py]

    def __getitem__(self, index):
        """__getitem__

        :param index:
        """
        img_path = self.files[self.split][index].rstrip()
        lbl_path = os.path.join(
            self.annotations_base,
            img_path.split(os.sep)[-2],
            os.path.basename(img_path)[:-15] + "gtFine_labelIds.png",
        )
        lbl_path = lbl_path.replace('leftgtFine', 'gtFine') # <- add this code for solving error
        # print('lbl_path =', lbl_path)
        name = img_path.split(os.sep)[-1][:-4] + ".png"

        img = Image.open(img_path)
        img = np.array(img, dtype=np.uint8)

        lbl = Image.open(lbl_path)
        lbl = self.encode_segmap(np.array(lbl, dtype=np.uint8))

        if self.augmentations is not None:
            img, lbl = self.augmentations(img, lbl)

        if self.is_transform:
            img, lbl = self.transform(img, lbl)


        return img, lbl, name

I share it for reference.
Perhaps my cityscapes data is out of date...

License?

Could you add a license? Is it free to use for commercial work?

ForkingPickler(file, protocol).dump(obj)BrokenPipeError: [Errno 32] Broken pipe" in training after 500 iters.

@PingoLH My windows10 with RTX2070 8G， pytorch1.2. When i trained the data for 500 iters, i run into a problem that

"ForkingPickler(file, protocol).dump(obj)BrokenPipeError: [Errno 32] Broken pipe"
i checked some blogs and i found if the num_work in DataLoader was set 0, the problem could be solved. However, i found that the num_workers has been set as 8 in cig. I change it to 0, but it would not go on at 500 iters. So i don't kown how to slove it. Do you have any idea about the issue?

Where is the test.py

Hello buddy.
Thank you for your implementation.

Unfortunately, i can't find the test.py anywhere.
Do you have any plan to implement it.

Transfer learning with different classes

I would like to take a pre-trained Cityscapes model for transfer learning with a dataset that does not have the same amount of classes. At the moment, this causes an error in the module method load_state_dict(). What do I have to change in the code to not initialize the last network layer for making it work?

"CUDA out of memory." at training.

Hi!
I try to train with this command.(At my windows PC with RTX2070)

F:\Users\sounansu\Anaconda3\FCHarDNet>\python train.py --config configs\hardnet.yml
.....
RuntimeError: CUDA out of memory. Tried to allocate 40.00 MiB (GPU 0; 8.00 GiB total capacity; 5.98 GiB already allocated; 24.97 MiB free; 30.09 MiB cached)

Please teach me how to modify hardnet.yml!

upsample problem

Can deconv replace bilinear upsample?

about background class

Hi, thanks for the great work !
I notice that you don not count background as a class, will this cause false positive classification on background pixels? Also in my experiment, it has only one class, how to suit for this case?

Test Pretrain model for single image

Thanks for an amazing repository.

How can I test the result with Cityscapes pre-trained weights for the single image?

Model architecture

Thank you for your implementation. I just wonder that where does the FC-HarDNet70 architecture come from as i don't see it being described in the paper?

How was weights/hardnet_petite_base.pth trained?

Hi, I'm looking to convert this repo to Tensorflow. I believe most of the operations here are standard and it should be relatively straightforward to convert to Tensorflow.

I will however need to regenerate the pretrained weights for hardnet_petite_base. I was wondering if you could share the details of how you trained this?

Is the structure, just the decoder portion of the network followed by an FC layer to the number of classes and then Trained on ImageNet for 100 epochs?

Is Cityscapes "mIOU" metric correspond to "Mean IoU" or "Overall Acc" in metrics.py

Is Cityscapes "mIOU" metric correspond to "Mean IoU" or "Overall Acc" in metrics.py
and what is about the loss value when the model converge?

validate problem

Hi, when I run the validate.py file using this commond

python validate.py --config=configs/hardnet.yml --model_path=./weights/hardnet70_cityscapes_model.pkl

I got this error, How to fix it, Thank you!

validate.py:237: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  cfg = yaml.load(fp)
Found 500 val images
Traceback (most recent call last):
  File "validate.py", line 239, in <module>
    validate(cfg, args)
  File "validate.py", line 56, in validate
    model.v2_transform(trt=False)
  File "/home/cjs/trunk_ws/segmentation/env_hardnet/FCHarDNet/ptsemseg/models/hardnet.py", line 327, in v2_transform
    self.base[i].transform(blk, trt)
  File "/home/cjs/trunk_ws/segmentation/env_hardnet/FCHarDNet/ptsemseg/models/hardnet.py", line 96, in transform
    self.conv_layers[i].weight[0:part[0], :, :,:] = w_src[:, 0:in_ch, :,:]
RuntimeError: a leaf Variable that requires grad has been used in an in-place operation.

Merge several classes into one class while training

Hello, thank you for sharing the great works.

I now stumbled on merging the classes while training.
I have original CItyscapes data and during the training, I want to merge some classes into one class.
Let's say class labels 2 to 10 merging to class 5 , and class 11, 12 to 15 merging to 11 for example.

Can you tell me how to implement what I want to do?

Thank you

Bug in _bootstrap_xentropy_single

In loss.py in function _bootstrap_xentropy_single, there is a bug in the following code portion :

        if sorted_loss[K] > thresh:  # the bug is here, it should be `sorted_loss[K] <= thresh`
            loss = sorted_loss[sorted_loss > thresh]
        else:
            loss = sorted_loss[:K]

Because with your current code, if the Kth loss value is below threshold it will be kept (else clause), whereas you don't want that.

why MIoU can not reach 77 in cityscapes val dataset?

Thanks for your great work;but when I use your pretrained model to validate,the MIoU is 70.19536 (hardnet70_cityscapes_model.pkl)and 71.179(hardnet70_cityscapes_model_2.pkl) respectively.So I want to ask is this the best model? Or do I need to do anything else? thanks!

Why does deconvolution loss function converge more slowly than bilinear interpolation？

Excuse me, why the image is resized in the training phase but not in the test phase. Is there any intention?

details on FC-HarDNet 68,76,84

Hello,
I appreciate for your kind responses for every questions!
now I'm working on the implementations of all hardnet models, but I couldn't find the details of the "first_channel" and "ch_list" on your repo for FC-HarDNet 68,76,84. your paper only describes on BLK depth, Growth rate, m in table 3.

For example, you specified first_ch = [32, 64] / ch_list = [ 128, 256, 320, 640, 1024] for HarDNet68.
first_ch = [16,24,32,48] / ch_list = [ 64, 96, 160, 224, 320] for FC-HarDNet70.

Could you please tell me the how many convolution layers are in front of FC-HarDNet and ch_list?

inference speed is much slower than as stated

Hi, thank you for the work !
I have test on my own segmentation task, with image input size 15361536, the inference speed on my 1080ti is about 300ms per image, which is far slower than 53fps with input size 10242048 as stated. Have you used tensorrt or any acceleration method ?

.

.

Code for BiseNet

Could you also share your code for BiseNet? Thanks.

Training unbalance on different GPUs?

I used 8 gpus to train the model, but most memory is placed on the first GPU and i can not fully utilize other gpus, is threre any solution? thanks!

Per-category IOU

Hi !! Could you show the per-category IoU results in Cityscape?

raise ValueError("Segmentation map contained invalid class values")

when i test the mode: python validate.py --model_path weights/hardnet70_cityscapes_model.pkl

raise self.exc_type(msg)

ValueError: Caught ValueError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/worker.py", line 185, in _worker_loop
data = fetcher.fetch(index)
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/yijiahe/AI/FCHarDNet/ptsemseg/loader/cityscapes_loader.py", line 170, in getitem
img, lbl = self.transform(img, lbl)
File "/home/yijiahe/AI/FCHarDNet/ptsemseg/loader/cityscapes_loader.py", line 209, in transform
raise ValueError("Segmentation map contained invalid class values")
ValueError: Segmentation map contained invalid class values

Convert PyTorch model to TensorRT

Hi @PingoLH

I am trying to convert the HarDNet pytorch segmentation model into a TensorRT engine
to speedup the process, I want to use the model on a Jetson Xavier device.

Could you please give some advice or some guidance for the conversion process?

Train sunrgbd data set to report errors

149it [01:19, 1.85it/s][0 1 2 5 7]
[0 1 2 5 7]
[ 0 1 2 5 14]
[ 0 1 2 5 14]
[ 0 1 2 5 8 9 14]
[ 0 1 2 5 9 14]
Traceback (most recent call last):
File "E:/vSLAM/FCHarDNet-master/train.py", line 269, in
train(cfg, writer, logger)
File "E:/vSLAM/FCHarDNet-master/train.py", line 183, in train
for i_val, (images_val, labels_val) in tqdm(enumerate(valloader)):
File "D:\Anaconda3\envs\pytorch1\lib\site-packages\tqdm\std.py", line 1102, in iter
for obj in iterable:
File "D:\Anaconda3\envs\pytorch1\lib\site-packages\torch\utils\data\dataloader.py", line 560, in next
batch = self.collate_fn([self.dataset[i] for i in indices])
File "D:\Anaconda3\envs\pytorch1\lib\site-packages\torch\utils\data\dataloader.py", line 560, in
batch = self.collate_fn([self.dataset[i] for i in indices])
File "E:\vSLAM\FCHarDNet-master\ptsemseg\loader\sunrgbd_loader.py", line 81, in getitem
img, lbl = self.transform(img, lbl)
File "E:\vSLAM\FCHarDNet-master\ptsemseg\loader\sunrgbd_loader.py", line 106, in transform
assert np.all(classes == classes1)
AssertionError

Training sunrgbd data set error why is the category not correct here?Is floating-point 64 changed to 32 by default?

Training protocol (specification for training settings)

Hi, Thanks for great contributions on the segmentation problem.
Now, I'm working to follow up your results in my environments as the same as the file that you have uploaded to 'Cityscapes pretrained weights' (77.7% mIOUs on validation set ). The same settings with the 'hardnet.yml' file perform only 76.8%. (not 77.7% )

I need more details on training. Could you please tell me regarding on this?

for example,

1. I need
   **# of GPUs, initial learning rate, weight decay, iterations, batch size, crop size you used for your records. (77.7%) **

2.Unfortunately, I have only 4 GPUs and could you tell me the results on 4 GPUs settings if you have any please?

3. Have you used the GPU parallel what you answered #8 ? If you did, could you please the way please to get the 77.7% results ? (same as the code in the issue?)

Error training using Pascal VOC

I've been trying to train the model Pascal VOC 2012 and SBD and these things keep popping up whenever the training process reaches the validation loop (the training part is perfectly fine).

C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [127,0,0], thread: [868,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [124,0,0], thread: [160,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [141,0,0], thread: [166,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [122,0,0], thread: [3,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [137,0,0], thread: [67,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [135,0,0], thread: [167,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [135,0,0], thread: [168,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [141,0,0], thread: [150,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [133,0,0], thread: [167,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [133,0,0], thread: [168,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [137,0,0], thread: [717,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [128,0,0], thread: [868,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [127,0,0], thread: [798,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [139,0,0], thread: [716,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [122,0,0], thread: [154,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [135,0,0], thread: [727,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [137,0,0], thread: [761,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [139,0,0], thread: [745,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [139,0,0], thread: [753,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [107,0,0], thread: [16,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [107,0,0], thread: [17,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [143,0,0], thread: [164,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [133,0,0], thread: [728,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [113,0,0], thread: [42,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [96,0,0], thread: [3,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [96,0,0], thread: [4,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [96,0,0], thread: [6,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [109,0,0], thread: [44,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [109,0,0], thread: [51,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [109,0,0], thread: [947,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [109,0,0], thread: [948,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [118,0,0], thread: [937,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [131,0,0], thread: [796,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [131,0,0], thread: [732,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [100,0,0], thread: [12,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [134,0,0], thread: [167,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [134,0,0], thread: [168,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [138,0,0], thread: [717,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [128,0,0], thread: [798,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [99,0,0], thread: [1023,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [118,0,0], thread: [32,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [118,0,0], thread: [33,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [118,0,0], thread: [57,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [121,0,0], thread: [154,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [100,0,0], thread: [1023,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [125,0,0], thread: [160,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [103,0,0], thread: [14,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [130,0,0], thread: [796,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [130,0,0], thread: [732,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [140,0,0], thread: [150,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [132,0,0], thread: [731,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [138,0,0], thread: [126,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [97,0,0], thread: [3,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [97,0,0], thread: [4,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [97,0,0], thread: [6,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [121,0,0], thread: [36,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [121,0,0], thread: [37,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [121,0,0], thread: [59,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [140,0,0], thread: [745,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [140,0,0], thread: [753,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [115,0,0], thread: [939,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [115,0,0], thread: [940,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [123,0,0], thread: [3,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [105,0,0], thread: [46,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [105,0,0], thread: [47,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [112,0,0], thread: [943,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [112,0,0], thread: [944,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [106,0,0], thread: [16,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [106,0,0], thread: [17,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [112,0,0], thread: [42,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [112,0,0], thread: [53,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [106,0,0], thread: [46,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [106,0,0], thread: [47,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [115,0,0], thread: [55,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [103,0,0], thread: [1021,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [144,0,0], thread: [163,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [106,0,0], thread: [951,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [106,0,0], thread: [952,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [136,0,0], thread: [761,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [125,0,0], thread: [63,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.
C:/cb/pytorch_1000000000000/work/aten/src/THCUNN/ClassNLLCriterion.cu:59: block: [136,0,0], thread: [125,0,0] Assertion `cur_target >= 0 && cur_target < n_classes` failed.

The config is mostly just default, I didn't change much aside from the image size. I also supplied the "sdb_path" to the dataloaders.

EDIT: It seems to be caused by the lack of augmentation in valloader. I tried again with augementation and it works fine for some reasons

Error on test

Hello,

When I use this command to test I got error:

python validate.py --config configs\hardnet.yml --model_path weights\hardnet70_cityscapes_model.pkl

Error:
File "validate.py", line 12, in
from pytorch_bn_fusion.bn_fusion import fuse_bn_recursively
ModuleNotFoundError: No module named 'pytorch_bn_fusion.bn_fusion'

torch verison: 1.1.0,

What is the difference between HarDBlock and HarDBlock_v2?

In the source code, only 'validate.py' uses HarDBlock_v2.
And the model with HarDBlock_v2 is faster than that with HarDBlock.
I want to know the difference between v1 and v2.

Is there a PTH trained based on the sun-rgbd data set?

Low GPU usage and low FPS.

Hi @PingoLH

I have changed test.py to run on a video. See implementation below:

device, model = init_model(args)
 proc_size = eval(args.size)

 cap = cv2.VideoCapture("/home/sieuwe/Downloads/Garmin Dash Cam 55 Video Sample (Highway) _ 1080p 30fps (HDR).mp4")

 while(True):

     begin = time.perf_counter()
     ret, frame = cap.read()
     img_raw, decoded = process_img(frame, proc_size, device, model)
     cv2.imshow("OUT",decoded)
     cv2.imshow("IN",img_raw)
     end = time.perf_counter()
     print("fps", 1 / (end - begin))

     if cv2.waitKey(1) & 0xFF == ord('q'):
         break

 cap.release()
 cv2.destroyAllWindows()

I am running a GTX1080 and thus expected a FPS of around 40. I however got a much lower fps of 21. When looking at my GPU util in nvidia-smi I see that it only uses 10% of my gpu. How can I increase the GPU util to get higher FPS.

Using CUDA 10.2 on Ubunut 18.04 with pytorch version 1.4.0

Thanks for your help

Sieuwe

please help me! I meet a problem when training my own dataset

I have a dataset with 14 classes(include 13 kinds object classes and 1 kind background)
In the dataset, background'id is set to 0, and the others classes' id is 1-13,
in the label picture, the background pix value is 0, the others classes pix value is 1-13
The problem is that, when training the model, the IoU of class 1 is always 0.0,
Overall Acc: 0.7006523041507537
INFO:ptsemseg:Overall Acc: : 0.7006523041507537
Mean Acc : 0.2108480790158249
INFO:ptsemseg:Mean Acc : : 0.2108480790158249
FreqW Acc : 0.5198288277808017
INFO:ptsemseg:FreqW Acc : : 0.5198288277808017
Mean IoU : 0.14262183776063359
INFO:ptsemseg:Mean IoU : : 0.14262183776063359
INFO:ptsemseg:0: 0.6279548955918839
INFO:ptsemseg:1: 0.0
INFO:ptsemseg:2: 0.0
INFO:ptsemseg:3: 0.6441913898204116
INFO:ptsemseg:4: 0.5818516148968814
INFO:ptsemseg:5: 1.0196236772931845e-06
INFO:ptsemseg:6: 0.0
INFO:ptsemseg:7: 0.0
INFO:ptsemseg:8: 0.0
INFO:ptsemseg:9: 0.0
INFO:ptsemseg:10: 8.497095538252379e-05
INFO:ptsemseg:11: 0.0
INFO:ptsemseg:12: 0.0

futhermore, sometimes it get error:
INFO:ptsemseg:Iter 1500 Val Loss: 1.6997
/home/new02/yh/FCHarDNet/ptsemseg/metrics.py:34: RuntimeWarning: invalid value encountered in true_divide
iu = np.diag(hist) / (hist.sum(axis=1) + hist.sum(axis=0) - np.diag(hist))
Overall Acc: 0.6941342720781378
INFO:ptsemseg:Overall Acc: : 0.6941342720781378
Mean Acc : 0.20532103027188917
INFO:ptsemseg:Mean Acc : : 0.20532103027188917
FreqW Acc : 0.5145511719663355
INFO:ptsemseg:FreqW Acc : : 0.5145511719663355
Mean IoU : 0.15183979860183058
INFO:ptsemseg:Mean IoU : : 0.15183979860183058
INFO:ptsemseg:0: 0.6409373749170948
INFO:ptsemseg:1: nan
WARNING:root:NaN or Inf found in input tensor.
INFO:ptsemseg:2: 0.0
INFO:ptsemseg:3: 0.6054743860395182
INFO:ptsemseg:4: 0.5756210137925749
INFO:ptsemseg:5: 1.0195457108221718e-06
INFO:ptsemseg:6: 0.0
INFO:ptsemseg:7: 0.0
INFO:ptsemseg:8: 0.0
INFO:ptsemseg:9: 0.0
INFO:ptsemseg:10: 4.378892706825406e-05
INFO:ptsemseg:11: 0.0
INFO:ptsemseg:12: 0.0
……
INFO:ptsemseg:Iter 1800 Val Loss: 1.3872
Overall Acc: 0.7570247568021282
INFO:ptsemseg:Overall Acc: : 0.7570247568021282
Mean Acc : 0.22404660812178812
INFO:ptsemseg:Mean Acc : : 0.22404660812178812
FreqW Acc : 0.59153389710979
INFO:ptsemseg:FreqW Acc : : 0.59153389710979
Mean IoU : 0.17551320192855324
INFO:ptsemseg:Mean IoU : : 0.17551320192855324
INFO:ptsemseg:0: 0.733411667979755
INFO:ptsemseg:1: nan
WARNING:root:NaN or Inf found in input tensor.
INFO:ptsemseg:2: 0.0
INFO:ptsemseg:3: 0.7271575023121397
INFO:ptsemseg:4: 0.6455313922516163
INFO:ptsemseg:5: 0.0
INFO:ptsemseg:6: 0.0
INFO:ptsemseg:7: 0.0
INFO:ptsemseg:8: 0.0
INFO:ptsemseg:9: 0.0
INFO:ptsemseg:10: 5.406602233699094e-05
INFO:ptsemseg:11: 3.7945767908505163e-06
INFO:ptsemseg:12: 0.0

I modified the cityscapes_loader.py:
colors = [ #[ 0, 0, 0],
[128, 64, 128],
[244, 35, 232],
[70, 70, 70],
[102, 102, 156],
[190, 153, 153],
[153, 153, 153],
[250, 170, 30],
[220, 220, 0],
[107, 142, 35],
[152, 251, 152],
[0, 130, 180],
[220, 20, 60],
[255, 0, 0],
#[0, 0, 142],
#[0, 0, 70],
#[0, 60, 100],
#[0, 80, 100],
#[0, 0, 230],
#[119, 11, 32],
]

label_colours = dict(zip(range(13), colors))

   self.void_classes = [0]
    self.valid_classes = [
        1,
        2,
        3,
        4,
        5,
        6,
        7,
        8,
        9,
        10,
        11,
        12,
        13,
    ]

    self.class_names = [
        "background",
        "class1",
        "class2",
        "class3",
        "class4",
        "class5",
        "class6",
        "class7",
        "class8",
        "class9",
        "class10",
        "class12",
        "class13",
    ]

    self.ignore_index = 250
    self.class_map = dict(zip(self.valid_classes, range(13)))

please help me! How can I fix the problem?

Data transform

@PingoLH I'm a little confused about the data preprocessing part, how did you determine the mean and standard deviation in tranforming. The code you use just like followes:

    value_scale = 255
    mean = [0.406, 0.456, 0.485]
    mean = [item * value_scale for item in mean]
    std = [0.225, 0.224, 0.229]
    std = [item * value_scale for item in std]

And if i change another dataset, how should i do to determine the mean and std?

GFLOPs, GMACs, fps measurement

Hi, Could you please share the codes to measure about Gflops/GMACs and CIO please?
(I got the results of fps and parameters, but I couldn't find the codes to compute gflops and cio )

源码中是否有继续训练的code，不用每次从头train？

Problems about bootstrapped CE loss

Hi, I reference your HarDNet structure to train cityscapes (on my own segmentation codebase, which could match the performance of almost all popular segmentation networks).
I use your pre-trained ImageNet network and follow almost all configs (except l use CE loss) as you set in this repo, but just got a 70.49 MIOU on the validation set (single scale inference).
But when I try to use the bootstrapped CE loss you implemented, I find that the loss is always in a very small range (around 0.15) and training pixel accuracy does not have a smooth increase. Is it normal when you train your model?
I'll appreciate it if you answer this question!