This repository is for the CVPR2023 paper "Learning Orthogonal Prototypes for Generalized Few-shot Semantic Segmentation".
The code is verified with Python 3.6 and PyTorch 1.8. It also relies on NumPy and OpenCV.
Please refer to PFENet to get PASCAL VOC with SBD and COCO 2014. The ground truth images for COCO 2014 can be generated using utils/coco_parse_script.py. Data paths should be as follows:
.{YOUR_PASCAL_PATH}
├── JPEGImages
├── SegmentationClassAug
.{YOUR_COCO_PATH}
├── train2014
├── train_no_crowd
├── val2014
├── val_no_crowd
After the first stage training, the code will save the filenames for each class as txt files in dataset/list/. Then, you can use utils/gen_fs_list.py to select few-shot data for the second stage training with different random seeds.
Download ImageNet pretrained ResNet-50 here. Check models to get pretrained POP models.
POP performs two-stage training for generalized few-shot semantic segmentation.
We use 1 GPU to train a base model on PASCAL-5i and 4 GPUs on COCO-20i. Run the training code with scripts/train_coco_fold0_base_q.sh and scripts/train_voc_fold0_base.sh. You should modify the arguments according to your settings. Note that you should specify YOUR_RESTORE_PATH as the path to ImageNet-pretrained ResNet-50 models.
Run the training code with scripts/ft_coco.sh and scripts/ft_voc.sh with modified arguments according to your settings. While ft_pop.py also supports larger batch size with multi-gpu training, we find small batch size often works better. Note that you should specify YOUR_RESTORE_PATH as the path to the base models trained in Stage 1. You can also finetune them on different support data by setting multiple random seeds.
To additionally speed up the training process, we support mix precision training in train_base.py and ft_pop.py, which can significantly reduce the time cost. To enable it, you can add --fp16 in the training scripts.
Use scripts/evaluate_coco_fold0.sh and scripts/evaluate_voc_fold0.sh to perform GFSS evaluation on the two datasets.
This repo is mainly built based on PFENet and GFS-Seg. Thanks for their great work!
If you find our code useful, please consider to cite with:
@inproceedings{liu2023learning,
title={Learning Orthogonal Prototypes for Generalized Few-shot Semantic Segmentation},
author={Liu, Sun-Ao and Zhang, Yiheng and Qiu, Zhaofan and Xie, Hongtao and Zhang, Yongdong and Yao, Ting},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
year={2023}
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent