deasonyuan / udat

Unsupervised Domain Adaptation for Nighttime Aerial Tracking (CVPR2022)

Unsupervised Domain Adaptation for Nighttime Aerial Tracking (CVPR2022)

Junjie Ye, Changhong Fu, Guangze Zheng, Danda Pani Paudel, and Guang Chen. Unsupervised Domain Adaptation for Nighttime Aerial Tracking. In CVPR, pages 1-10, 2022.

Overview

UDAT is an unsupervised domain adaptation framework for visual object tracking. This repo contains its Python implementation.

Paper | NAT2021 benchmark

Testing UDAT

1. Preprocessing

Before training, we need to preprocess the unlabelled training data to generate training pairs.

1. Download the proposed NAT2021-train set

2. Customize the directory of the train set in lowlight_enhancement.py and enhance the nighttime sequences

   cd preprocessing/
   python lowlight_enhancement.py # enhanced sequences will be saved at '/YOUR/PATH/NAT2021/train/data_seq_enhanced/'

3. Download the video saliency detection model here and place it at preprocessing/models/checkpoints/.

4. Predict salient objects and obtain candidate boxes

   python inference.py # candidate boxes will be saved at 'coarse_boxes/' as .npy

5. Generate pseudo annotations from candidate boxes using dynamic programming

   python gen_seq_bboxes.py # pseudo box sequences will be saved at 'pseudo_anno/'

6. Generate cropped training patches and a JSON file for training

   python par_crop.py
   python gen_json.py

2. Train

Take UDAT-CAR for instance.

1. Apart from above target domain dataset NAT2021, you need to download and prepare source domain datasets VID and GOT-10K.

2. Download the pre-trained daytime model (SiamCAR/SiamBAN) and place it at UDAT/tools/snapshot.

3. Start training

   cd UDAT/CAR
   export PYTHONPATH=$PWD
   python tools/train.py

3. Test

Take UDAT-CAR for instance.

1. For quick test, you can download our trained model for UDAT-CAR (or UDAT-BAN) and place it at UDAT/CAR/experiments/udatcar_r50_l234.

2. Start testing

   python tools/test.py --dataset NAT

4. Eval

1. Start evaluating

   python tools/eval.py --dataset NAT

Demo

Reference

@Inproceedings{Ye2022CVPR,

title={{Unsupervised Domain Adaptation for Nighttime Aerial Tracking}},

author={Ye, Junjie and Fu, Changhong and Zheng, Guangze and Paudel, Danda Pani and Chen, Guang},

booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},

year={2022},

pages={1-10}

}

Acknowledgments

We sincerely thank the contribution of following repos: SiamCAR, SiamBAN, DCFNet, DCE, and USOT.

Contact

If you have any questions, please contact Junjie Ye at [email protected] or Changhong Fu at [email protected].