shenyunhang / perceptron

HUWSOD: Holistic Self-training for Unified Weakly Supervised Object Detection

By Liujuan Cao, Jianghang Lin, Zebo Hong, Yunhang Shen, Shaohui Lin, Chao Chen, Rongrong Ji.

This project is based on Detectron2.

Installation

Install Detectron2 following the instructions.

Install HUWSOD project:

pip3 install -r WSL/requirements.txt
git submodule update --init --recursive

python3 -m pip install -e detectron2
python3 -m pip install -e WSL/

Dataset Preparation

PASCAL VOC

Please follow this to creating symlinks for PASCAL VOC.

[This is Optional] PASCAL VOC Proposal

Download MCG proposal from here to datasets/proposals, and transform it to pickle serialization format:

cd datasets/proposals
tar xvzf MCG-Pascal-Main_trainvaltest_2007-boxes.tgz
tar xvzf MCG-Pascal-Main_trainvaltest_2012-boxes.tgz
cd ../../
python3 WSL/tools/proposal_convert.py voc_2007_train datasets/proposals/MCG-Pascal-Main_trainvaltest_2007-boxes datasets/proposals/mcg_voc_2007_train_d2.pkl
python3 WSL/tools/proposal_convert.py voc_2007_val datasets/proposals/MCG-Pascal-Main_trainvaltest_2007-boxes datasets/proposals/mcg_voc_2007_val_d2.pkl
python3 WSL/tools/proposal_convert.py voc_2007_test datasets/proposals/MCG-Pascal-Main_trainvaltest_2007-boxes datasets/proposals/mcg_voc_2007_test_d2.pkl
python3 WSL/tools/proposal_convert.py voc_2012_train datasets/proposals/MCG-Pascal-Main_trainvaltest_2012-boxes datasets/proposals/mcg_voc_2012_train_d2.pkl
python3 WSL/tools/proposal_convert.py voc_2012_val datasets/proposals/MCG-Pascal-Main_trainvaltest_2012-boxes datasets/proposals/mcg_voc_2012_val_d2.pkl
python3 WSL/tools/proposal_convert.py voc_2012_test datasets/proposals/MCG-Pascal-Main_trainvaltest_2012-boxes datasets/proposals/mcg_voc_2012_test_d2.pkl

COCO:

Please follow this to creating symlinks for MS COCO.

Download

wget https://dl.fbaipublicfiles.com/detectron/coco/coco_annotations_minival.tgz

Please follow this to download minival and valminusminival annotations.

[This is Optional] COCO Proposal

Download MCG proposal from here to datasets/proposals, and transform it to pickle serialization format:

cd datasets/proposals
tar xvzf MCG-COCO-train2014-boxes.tgz
tar xvzf MCG-COCO-val2014-boxes.tgz
cd ../../
python3 WSL/tools/proposal_convert.py coco_2014_train datasets/proposals/MCG-COCO-train2014-boxes datasets/proposals/mcg_coco_2014_train_d2.pkl
python3 WSL/tools/proposal_convert.py coco_2014_valminusminival datasets/proposals/MCG-COCO-val2014-boxes datasets/proposals/mcg_coco_2014_valminusminival_d2.pkl
python3 WSL/tools/proposal_convert.py coco_2014_minival datasets/proposals/MCG-COCO-val2014-boxes datasets/proposals/mcg_coco_2014_minival_d2.pkl

Model Preparation

Download models from this here:

mv models $perceptron

After dataset and model preparation, we have the following directory structure:

perceptron
├── datasets
│   ├── coco
│   ├── VOC2007
│   ├── VOC2012
│   ├── VOCdevkit
│   ├── proposals
│   │   ├── mcg_coco_2014_minival_d2.pkl
│   │   ├── mcg_coco_2014_train_d2.pkl
│   │   ├── mcg_coco_2014_valminusminival_d2.pkl
│   │   ├── MCG-COCO-train2014-boxes
│   │   ├── MCG-COCO-train2014-boxes.tgz
│   │   ├── MCG-COCO-val2014-boxes
│   │   ├── MCG-COCO-val2014-boxes.tgz
│   │   ├── MCG-Pascal-Main_trainvaltest_2007-boxes
│   │   ├── MCG-Pascal-Main_trainvaltest_2007-boxes.tgz
│   │   ├── MCG-Pascal-Main_trainvaltest_2012-boxes
│   │   ├── MCG-Pascal-Main_trainvaltest_2012-boxes.tgz
│   │   ├── mcg_voc_2007_test_d2.pkl
│   │   ├── mcg_voc_2007_train_d2.pkl
│   │   ├── mcg_voc_2007_val_d2.pkl
│   │   ├── mcg_voc_2012_train_d2.pkl
│   │   ├── mcg_voc_2012_val_d2.pkl
│   │   ├── mcg_voc_2012_val_instance_d2.pkl
├── models
│   ├── DRN-WSOD
│   │   ├── resnet101_ws_model_120_d2.pkl
│   │   ├── resnet101_ws_model_120.pkl
│   │   ├── resnet18_ws_model_120_d2.pkl
│   │   ├── resnet18_ws_model_120.pkl
│   │   ├── resnet50_ws_model_120_d2.pkl
│   │   ├── resnet50_ws_model_120.pkl
│   │   ├── densenet121_ws_model_120.pkl
│   └── VGG
│       └── VGG_ILSVRC_16_layers_v1_d2.pkl
├── detectron2
    ├── ...
└── WSL
    ├── ...

Evaluation with trained models

Download trained models here and put them to output/.

To evaluate trained model, run

python3.9 WSL/tools/train_net_huwsod.py --dist-url "tcp://127.0.0.1:52051" --eval-only --num-gpus 8 --config-file WSL/configs/PascalVOC-Detection/huwsod_WSR_18_DC5_4x1_160e.yaml OUTPUT_DIR output/WSL/configs/PascalVOC-Detection/huwsod_WSR_18_DC5_4x1_160e_20220318 MODEL.WEIGHTS output/WSL/configs/PascalVOC-Detection/huwsod_WSR_18_DC5_4x1_160e_20220318/model_final.pth

With proposal on image (poi):

python3.9 WSL/tools/train_net_huwsod.py --dist-url "tcp://127.0.0.1:52050" --eval-only --num-gpus 8 --config-file WSL/configs/PascalVOC-Detection/huwsod_poi_WSR_18_DC5_4x1_160e.yaml OUTPUT_DIR output/WSL/configs/PascalVOC-Detection/huwsod_poi_WSR_18_DC5_4x1_160e_20220318/ MODEL.WEIGHTS output/WSL/configs/PascalVOC-Detection/huwsod_poi_WSR_18_DC5_4x1_160e_20220318/model_final.pth

Main results

	voc_2007_test mAP	voc_2007_tain CorLoc	voc_2007_val CorLoc
huwsod_WSR_18_DC5_4x1_160e_20220318	53.3611	71.8983	71.4805
huwsod_poi_WSR_18_DC5_4x1_160e_20220318	59.4654	75.0364	75.0817

Traning

python3.9 WSL/tools/train_net_huwsod.py --dist-url "tcp://127.0.0.1:52053" --num-gpus 4 --config-file WSL/configs/PascalVOC-Detection/huwsod_WSR_18_DC5_4x1_160e.yaml OUTPUT_DIR output/WSL/configs/PascalVOC-Detection/huwsod_WSR_18_DC5_4x1_160e_`date +'%Y%m%d_%H%M%S'`

With proposal on image (poi):

python3.9 WSL/tools/train_net_huwsod.py --dist-url "tcp://127.0.0.1:52054" --num-gpus 4 --config-file WSL/configs/PascalVOC-Detection/huwsod_poi_WSR_18_DC5_4x1_160e.yaml OUTPUT_DIR output/WSL/configs/PascalVOC-Detection/huwsod_poi_WSR_18_DC5_4x1_160e_`date +'%Y%m%d_%H%M%S'`

Using Other WSOD mthods

WSDDN

ResNet18-WS

python3.9 WSL/tools/train_net.py --dist-url "tcp://127.0.0.1:52054" --num-gpus 4 --config-file WSL/configs/PascalVOC-Detection/wsddn_WSR_18_DC5.yaml OUTPUT_DIR output/WSL/configs/PascalVOC-Detection/wsddn_WSR_18_DC5_`date +'%Y%m%d_%H%M%S'`

ResNet50-WS

python3.9 WSL/tools/train_net.py --dist-url "tcp://127.0.0.1:52054" --num-gpus 4 --config-file WSL/configs/PascalVOC-Detection/wsddn_WSR_50_DC5.yaml OUTPUT_DIR output/WSL/configs/PascalVOC-Detection/wsddn_WSR_50_DC5_`date +'%Y%m%d_%H%M%S'`

ResNet101-WS

python3.9 WSL/tools/train_net.py --dist-url "tcp://127.0.0.1:52054" --num-gpus 4 --config-file WSL/configs/PascalVOC-Detection/wsddn_WSR_101_DC5.yaml OUTPUT_DIR output/WSL/configs/PascalVOC-Detection/wsddn_WSR_101_DC5_`date +'%Y%m%d_%H%M%S'`

VGG16

python3.9 WSL/tools/train_net.py --dist-url "tcp://127.0.0.1:52054" --num-gpus 4 --config-file WSL/configs/PascalVOC-Detection/wsddn_V_16_DC5.yaml OUTPUT_DIR output/WSL/configs/PascalVOC-Detection/wsddn_V_16_DC5_`date +'%Y%m%d_%H%M%S'`

OICR

ResNet18-WS

python3.9 WSL/tools/train_net.py --dist-url "tcp://127.0.0.1:52054" --num-gpus 4 --config-file WSL/configs/PascalVOC-Detection/oicr_WSR_18_DC5.yaml OUTPUT_DIR output/WSL/configs/PascalVOC-Detection/oicr_WSR_18_DC5_`date +'%Y%m%d_%H%M%S'`

ResNet50-WS

python3.9 WSL/tools/train_net.py --dist-url "tcp://127.0.0.1:52054" --num-gpus 4 --config-file WSL/configs/PascalVOC-Detection/oicr_WSR_50_DC5.yaml OUTPUT_DIR output/WSL/configs/PascalVOC-Detection/oicr_WSR_50_DC5_`date +'%Y%m%d_%H%M%S'`

ResNet101-WS

python3.9 WSL/tools/train_net.py --dist-url "tcp://127.0.0.1:52054" --num-gpus 4 --config-file WSL/configs/PascalVOC-Detection/oicr_WSR_101_DC5.yaml OUTPUT_DIR output/WSL/configs/PascalVOC-Detection/oicr_WSR_101_DC5_`date +'%Y%m%d_%H%M%S'`

VGG16

python3.9 WSL/tools/train_net.py --dist-url "tcp://127.0.0.1:52054" --num-gpus 4 --config-file WSL/configs/PascalVOC-Detection/oicr_V_16_DC5.yaml OUTPUT_DIR output/WSL/configs/PascalVOC-Detection/oicr_V_16_DC5_`date +'%Y%m%d_%H%M%S'`

perceptron's Issues

Links to create symlinks for datasets are broken

Hello. Thanks for such great work.

I am trying to re-create your results but the links provided for setting up the datasets are broken. Also, can I train this for my own dataset ?