Under Develop.
PASSL is a PAddle based vision library for state-of-the-art Self-Supervised Learning research. PASSL aims to accelerate research cycle in self-supervised learning: from designing a new self-supervised task to evaluating the learned representations.
- Install PaddlePaddle following the official install guidence.
- Download the ImageNet dataset from the link.
- Then, and move validation images to labeled subfolders, using the linked shell script
- Create a symlink under PASSL/data/ as the following examples:
ln -s $YOUR_ILSVRC2012_PATH data/ILSVRC2012
- At last, the folder looks like:
PASSL ├── configs ├── LICENSE ├── passl ├── tools ├── README.md └── data └── ILSVRC2012 ├── train └── val
Models are all trained with ResNet-50 backbone.
epochs | official results | passl results | Backbone | Model | |
---|---|---|---|---|---|
MoCo | 200 | 60.6 | 60.64 | ResNet-50 | download |
MoCo v2 | 200 | 67.7 | 67.72 | ResNet-50 | download |
python tools/train.py -c configs/moco_v[1,2]_r50.yaml --output_dir $SPECIFIED_OUTPUT_DIR
python tools/train.py -c configs/moco_v[1,2]_r50.yaml --num-gpus 8
Pretraining models with 200 epochs can be found at MoCo v1 and MoCo v2
Note: The default learning rate in config files is for 8 GPUs. If using differnt number GPUs, the total batch size will change in proportion, you have to scale the learning rate following new_lr = old_lr * new_ngpus / old_ngpus
. Replacing v[1,2] to v1 or v2 according to your requriement.
python tools/extract_weight.py $YOUR_TRAINED_MODEL_PATH --output $YOUR_EXTRACTED_BACKBONE_PATH
python tools/train.py -c configs/moco/moco_clas_r50.yaml --pretrained $YOUR_EXTRACTED_BACKBONE_PATH --num-gpus 8 --output_dir $SPECIFIED_OUTPUT_DIR
python tools/train.py -c configs/moco/moco_clas_r50.yaml --load $YOUR_TRAINED_CLS_MODEL --evaluate-only --num-gpus 8 --output_dir $SPECIFIED_OUTPUT_DIR
The trained linear weights in conjuction with the backbone weights can be found at MoCo v1 linear and MoCo v2 linear
python tools/train.py -c configs/byol/byol_r50_IM.yaml --num-gpus 4 --output_dir $SPECIFIED_OUTPUT_DIR
Note: The default learning rate in config files is for 4 GPUs. If using differnt number GPUs, the total batch size will change in proportion, you have to scale the learning rate following new_lr = old_lr * new_ngpus / old_ngpus
. Replacing v[1,2] to v1 or v2 according to your requriement.
python tools/extract_weight.py $YOUR_TRAINED_MODEL_PATH --output $YOUR_EXTRACTED_BACKBONE_PATH
python tools/train.py -c configs/byol/byol_r50_IM_clas.yaml --pretrained $YOUR_EXTRACTED_BACKBONE_PATH --num-gpus 4 --output_dir $SPECIFIED_OUTPUT_DIR
python tools/train.py -c configs/byol/byol_r50_IM_clas.yaml --load $YOUR_TRAINED_CLS_MODEL --evaluate-only --num-gpus 4 --output_dir $SPECIFIED_OUTPUT_DIR