caigentan / swinnet

SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detection

---

Pytorch implementation of the paper SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detection(The paper has been accepted by IEEE Transactions on Circuits and Systems for Video Technology. The details are in paper)，For more details, please(https://github.com/liuzywen/SwinNet)

Authors:Zhengyi Liu, Yacheng Tan, Qian He, Yun Xiao

---

Prerequisites

---

• 
• 
• 
• 
• 
• 
• 

Data Preparation

---

Put the raw data under the following directory:

─ datasets\
 |─ RGB-D\
      ├─ test\
            |─···
 |────├─ train\
            |─···
 |    └─ validation\
            |─···
 |─ RGB-T\
      ├─ Train\
            |─···
      ├─ test\
            |─···
      └─ validation\
            |─···

Training

---

• Downloading necessary data: swin_base_patch4_window12_384_22k.pth
• Put the Pretrained models under Pre_train\ directory.
• After you download training dataset, just run SwinNet_train.py to train our model.

Testing

---

• After you download all the pre-trained model and testing dataset, just run SwinNet_test.py to generate the final prediction map.
• evaluation_tools start from Saliency-Evaluation-Toolbox

Results

---

RGB-D

dataset	Smeasure ↑	aFmeasure ↑	Emeasure ↑	MAE ↓
NLPR	0.941	0.908	0.967	0.018
NJU2K	0.935	0.922	0.934	0.027
STERE	0.919	0.893	0.929	0.033
DES	0.945	0.926	0.980	0.016
SIP	0.911	0.912	0.943	0.035
DUT	0.949	0.944	0.968	0.020

RGB-T

dataset	Smeasure ↑	aFmeasure ↑	Emeasure ↑	MAE ↓
VT821	0.904	0.847	0.926	0.030
VT1000	0.938	0.896	0.947	0.018
VT5000	0.912	0.865	0.924	0.026

Saliency map

---

All of the saliency maps mentioned in the paper are available on GoogleDrive, OneDriver, BaiduPan-code:, AliyunDriver

Visual comparison

---

RGB-D Visual comparison

RGB-T Visual comparison

Citation

If you find this work or code useful, please cite:

@article{liu2022,
  author={Liu, Zhengyi and Tan, Yacheng and He, Qian and Xiao, Yun},
  journal={IEEE Transactions on Circuits and Systems for Video Technology}, 
  title={SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detection}, 
  volume={32},
  number={7},
  pages={4486--4497},
  year={2022}}
``