robotwithcv / ddrnet

The official implementation of "Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes"

The official implementation of "Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes"

Achieve state-of-the-art trade-off between accuracy and speed on cityscapes and camvid, without using inference acceleration (like tensorRT) and extra data (like Mapillary)!

The overall architecture of our methods.

The details of "Deep Aggregation Pyramid Pooling Module(DAPPM)".

Usage

Currently, this repo contains the model codes and pretrained models for classification and semantic segmentation. Our models are trained using this code base HRNet-Semantic-Segmentation-pytorch-v1.1.

For training and testing DDRNet, you can refer to DDRNet.pytorch, Segmentation-Pytorch, semantic-segmentation, PaddleSeg, deci.ai.

Notice

There are some basic training tricks you should employ to reproduce our results including class balance sample, ohem, crop size of 1024x1024. More details can be found in the paper. And there is usually some variation with Cityscapes val results of the same model, maybe about 1% mIoU.

Keep "align_corners=False" in all places if you want to use our pretrained models for evaluation directly.

Pretrained models

ImageNet

DDRNet_23_slim(top-1 error:29.8): googledrive

DDRNet_23_slim using timm library, maybe helpful to train on own datasets(top-1 error:26.3, trained with a batch size of 256, warmup, cosine learning rate, 300 epoches and label smoothing): googledrive

DDRNet_23(top-1 error:24.1): googledrive

DDRNet_39(top-1 error:22.7): googledrive

Cityscapes

DDRNet_23_slim(val mIoU:77.8): googledrive

DDRNet_23(val mIoU:79.5): googledrive

CamVid

Dataset can be downloaded from the link.

DDRNet_23_slim: googledrive

Comma10K

Methods	Val loss	FPS
UNet-EfficientNetB0	0.0495	35.6
UNet-EfficientNetB4	0.0462	18.0
STDC1-Seg	0.0482	92.0
STDC2-Seg	0.0448	73.0
DDRNet_23_slim	0.0448	166.8
DDRNet_23	0.0433	62.7
DDRNet_39	0.0428	36.3

Please refer to comma10k-baseline for train and test details. The FPS is tested with the method of our paper under the same conditions.

Results on Cityscapes server

DDRNet_23_slim: 77.4

DDRNet_23: 79.4

DDRNet_39: 80.4 81.9(multi-scale and flip)

DDRNet_39 1.5x: 82.4(multi-scale and flip)

Test Speed

Evaluate the inference speed on Cityscapes dataset.

python3 DDRNet_23_slim_eval_speed.py

DDRNet-23-slim can achieve above 130fps by using the tool.

Citation

If you find this repo is useful for your research, Please consider citing our paper:

@article{hong2021deep,
  title={Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes},
  author={Hong, Yuanduo and Pan, Huihui and Sun, Weichao and Jia, Yisong},
  journal={arXiv preprint arXiv:2101.06085},
  year={2021}
}

@article{pan2022deep,
  title={Deep Dual-Resolution Networks for Real-Time and Accurate Semantic Segmentation of Traffic Scenes},
  author={Pan, Huihui and Hong, Yuanduo and Sun, Weichao and Jia, Yisong},
  journal={IEEE Transactions on Intelligent Transportation Systems},
  year={2022},
  publisher={IEEE}
}