bupt-ai-cz / restnet

RestNet: Boosting Cross-Domain Few-Shot Segmentation with Residual Transformation Network

RestNet: Boosting Cross-Domain Few-Shot Segmentation with Residual Transformation Network

[Paper] [Checkpoint]

Datasets

• PASCAL VOC2012:

  Download PASCAL VOC2012 devkit (train/val data):

  wget http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar

  Download PASCAL VOC2012 SDS extended mask annotations from [Google Drive].

• FSS-1000:

  Home: https://github.com/HKUSTCV/FSS-1000

  Direct: https://drive.google.com/file/d/16TgqOeI_0P41Eh3jWQlxlRXG9KIqtMgI/view

• ISIC2018:

  Home: http://challenge2018.isic-archive.com

  Direct (must login): https://challenge.isic-archive.com/data#2018

• Chest X-ray:

  Home: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4256233/

  Direct: https://www.kaggle.com/datasets/nikhilpandey360/chest-xray-masks-and-labels

Training

PASCAL VOC

python train.py --backbone {vgg16, resnet50} 
                --fold 4 
                --benchmark pascal
                --lr 1e-3
                --bsz 20
                --logpath "your_experiment_name"

Testing

FSS-1000

python test.py --backbone {vgg16, resnet50} 
               --benchmark fss 
               --nshot {1, 5} 
               --load "path_to_trained_model/best_model.pt"

ISIC

python test.py --backbone {vgg16, resnet50} 
               --fold {0, 1, 2, 3} 
               --benchmark isic 
               --nshot {1, 5} 
               --load "path_to_trained_model/best_model.pt"

Chest X-ray

python test.py --backbone {vgg16, resnet50} 
               --benchmark lung 
               --nshot {1, 5} 
               --load "path_to_trained_model/best_model.pt"

Acknowledgement

The implementation is highly based on HSNet and PATNet.
Thank them for their great work.

Citation

@misc{huang2023restnet,
      title={RestNet: Boosting Cross-Domain Few-Shot Segmentation with Residual Transformation Network}, 
      author={Xinyang Huang and Chuang Zhu and Wenkai Chen},
      year={2023},
      eprint={2308.13469},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Contact

Xinyang Huang ([email protected])

If you have any questions, you can contact us directly.

restnet's Issues

Missing conv_4d_new.py

https://github.com/bupt-ai-cz/RestNet/blob/a66e9f39592c1ba9417d00877839d320878561b7/model/learner.py#L4C6-L4C6

About IRE and MPS

Is the IRE module used in all datasets? Or is it only used when target dataset is FSS-1000?
Is MPS only used in training or is it used in both training and inference?

Have the backbone of ViT or other transformer based methods been tested as your backbone?

Hello author,

I found that your backbone is based on Resnet, and now it's a very popular transformer like ViT-based models. So, I want to ask if you have tried and tested on the backbone of ViT. Can you provide some open links for ViT-based models applied on Cross-domain and few-shot segmentation?

test detail

Is this setting to 600 or 1200?

when will the codes be released

Hi author,

I want to ask when will this work be released? Can you provide an initial codes firstly?

About inference

RestNet/test.py

Line 27 in b1db083

pred_mask = model.module.predict_mask_nshot(batch, nshot=nshot)

I wonder if you also need to forward twice when testing?
It is not clear if there is an omission here

Could you provide the trained weights and inference code?