wennyxy / dino-mc

DINO-MC: Self-supervised Contrastive Learning for Remote Sensing Imagery with Multi-sized Local Crops

PyTorch implementation and pretrained models for DINO-MC and DINO-TP. For details, please see our paper.

Pretrained models

Our models are pre-trained on SeCo-100K, and we list their k-nn and linear probing accuracy on EuroSAT. You can download the full checkpoint of pre-trained model with training infomation as well as weights of teacher and student networks used for the downstream tasks.

model	arch	params	k-nn	linear	download
DINO-MC	ViT-S/8	21M	93.41%	94.09%	pre-trained ckpt
DINO-MC	ResNet-50	23M	93.94%	95.59%	pre-trained ckpt
DINO-MC	WRN-50	69M	95.65%	95.70%	pre-trained ckpt
DINO-MC	Swin-t	28M	93.22%	90.54%	pre-trained ckpt
DINO-TP	ViT-S/8	21M	93.15%	93.89%	pre-trained ckpt
DINO-TP	ResNet-50	23M	79.05%	86.70%	pre-trained ckpt
DINO-TP	WRN-50	69M	86.27%	88.15%	pre-trained ckpt
DINO-TP	Swin-t	28M	92.83%	91.94%	pre-trained ckpt

Training

Our codes refer to DINO and SeCo. If you want to pre-train DINO-MC based on your datasets:

python run_with_submitit.py --nodes 1 --ngpus 4 --arch vit_small --data_mode mc --data_path /path/to/dataset/train --output_dir /path/to/saving_dir

Fine-tuning

After pre-training, you can evaluate the representations on three end-to-end fine-tuning downstream tasks.

model	arch	EuroSAT	download
DINO	ViT-S/8	97.98%	EuroSAT
DINO-MC	ViT-S/8	98.15%	EuroSAT
DINO-MC	Swin-tiny	98.43%	EuroSAT
DINO-MC	ResNet-50	98.69%	EuroSAT
DINO-MC	WRN-50-2	98.78%	EuroSAT

model	arch	BigEarthNet-10%	download	BigEarthNet	download
DINO	ResNet-50	79.67%	BigEarthNet-10% ckpt	85.38%	BigEarthNet ckpt
DINO-TP	ResNet-50	80.10%	BigEarthNet-10% ckpt	85.20%	BigEarthNet ckpt
DINO-MC	ResNet-50	82.55%	BigEarthNet-10% ckpt	86.86%	BigEarthNet ckpt
DINO-MC	WRN-50-2	82.67%	BigEarthNet-10% ckpt	87.22%	BigEarthNet ckpt
DINO-MC	Swin-tiny	83.84%	BigEarthNet-10% ckpt	88.75%	BigEarthNet ckpt
DINO-MC	ViT-S/8	84.20%	BigEarthNet-10% ckpt	88.69%	BigEarthNet ckpt

model	arch	Pre.	Rec.	F1	download
DINO	ResNet-50	57.37	44.21	49.53	OSCD
DINO-MC	ResNet-50	51.94	54.04	52.46	OSCD
DINO-TP	ResNet-50	51.10	49.03	49.74	OSCD
DINO	WRN-50-2	53.58	52.28	52.41	OSCD
DINO-MC	WRN-50-2	49.99	56.81	52.70	OSCD
DINO-TP	WRN-50-2	55.77	47.30	50.61	OSCD

Citation

If you find this repository useful, please consider giving a star ⭐ and citation:

@misc{wanyan2023dinomc,
      title={DINO-MC: Self-supervised Contrastive Learning for Remote Sensing Imagery with Multi-sized Local Crops}, 
      author={Xinye Wanyan and Sachith Seneviratne and Shuchang Shen and Michael Kirley},
      year={2023},
      eprint={2303.06670},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

dino-mc's Issues

Add url

https://arxiv.org/abs/2303.06670

The pre-training model is wrong.

Hello author, your pre-training model seems to be vit_small_8 instead of vit_small_16.

how to train swin-tiny's model

Thank you for this great article. I want to know how to train swin-tiny's model. Looking at the part about swin_t in the code can't be executed successfully. Do you know what the situation is?

The Structure of the datasets

Hello,author!
Would you please show us what the structure of the datasets are!
Thanks!