gengz / siamesefc-pytorch-vot

PyTorch implementation of 'Fully-convolutional Siamese networks for object tracking', slightly better results

SiamFC-PyTorch-VOT

• This is the PyTorch (0.3.1) implementation of SiamFC tracker [1], which was originally implemented using MatConvNet [2].
• In this project, we obtain slightly better results on VOT-16 and VOT-17 dataset.
• This project is originally forked from HengLan's implementation, which is with PyTorch 0.4.0 for OTB dataset.

Why fork and tinker?

• Make small modificatoins for using with VOT toolkit. (Run into some errors when directly using Heng's implementation.)
• Make small changes for better convergence during training (in my case).

Goal

• Ready-to-go version for using with VOT toolkit.
• As a baseline for related Siamese Tracker re-implementation.
• Several design choices tested.
• A more compact implementation of SiamFC [1].
• Reproduce the results of SiamFC [1] in VOT-2016 challenge (SiamFC-A), and in VOT-2017 challenge.

Requirements

• Python 2.7.12

• Python-opencv 3.2.0

• PyTorch 0.3.1

• Numpy 1.14.2

• Other packages listed in requirements.txt

  The results using packages of other version than above not guaranteed.

Data curation

• Download ILSVRC15, and unzip it (let's assume that $ILSVRC2015_Root is the path to your ILSVRC2015)

• Move $ILSVRC2015_Root/Data/VID/val into $ILSVRC2015_Root/Data/VID/train/, so we have five sub-folders in $ILSVRC2015_Root/Data/VID/train/

• Move $ILSVRC2015_Root/Annotations/VID/val into $ILSVRC2015_Root/Annotations/VID/train/, so we have five sub-folders in $ILSVRC2015_Root/Annotations/VID/train/

• Generate image crops

  • cd $SiamFC-PyTorch/ILSVRC15-curation/ (Assume you've downloaded the rep and its path is $SiamFC-PyTorch)
  • change vid_curated_path in gen_image_crops_VID.py to save your crops
  • run $python gen_image_crops_VID.py, then you can check the cropped images in your saving path (i.e., vid_curated_path). It takes a day or two for image crops generation.

• Generate imdb for training and validation

  • cd $SiamFC-PyTorch/ILSVRC15-curation/
  • change vid_root_path and vid_curated_path to your custom path in gen_imdb_VID.py
  • run $python gen_imdb_VID.py, then you will get two json files imdb_video_train.json (~ 430MB) and imdb_video_val.json (~ 28MB) in current folder, which are used for training and validation.

Train

• cd $SiamFC-PyTorch/Train/
• Change data_dir, train_imdb and val_imdb to your custom cropping path, training and validation json files.
• run $python run_Train_SiamFC.py
• some notes for training:
  • the options for training are in Config.py
  • each epoch (50 in total) may take 6 minuts (Nvidia Titan Pascal, num_worker=8 in my case)

Tracking

• cd $SiamFC-PyTorch/Tracking/
• Take a look at Config.py first, which contains all parameters for tracking
• Change self.net_base_path to the path saving your trained models
• Change self.net to indicate whcih model you want for evaluation, and I've uploaded a trained model SiamFC_45_model.pth in this rep (located in $SiamFC-PyTorch/Train/model/)
• The default parameters I use for my results is as listed in Config.py.
• Copy all the files under $SiamFC-PyTorch/Train/matlab to $VOT-Workspace. And modify paths in all those files. (Don't panic, just few lines in each file.)
• Run VOT evaluation as described in VOT toolkit documentation.

Results

Compare EAO:

dataset	result of this repo	result in vot paper
VOT 16	0.24	0.24
VOT 17	0.20	0.19

References

[1] L. Bertinetto, J. Valmadre, J. F. Henriques, A. Vedaldi, and P. H. Torr. Fully-convolutional siamese networks for object tracking. In ECCV Workshop, 2016.

[2] A. Vedaldi and K. Lenc. Matconvnet – convolutional neural networks for matlab. In ACM MM, 2015.

[3]https://github.com/HengLan/SiamFC-PyTorch

siamesefc-pytorch-vot's Issues

EAO is lower than report

I follow the same data preparation procedure and run the code, but only get 0.1801 EAO on VOT17. So i'm wondering is there any other issues that i should notice

a small terminology query

I am little confused in tracking part where you make pyramid scales of crops?

Is there necessity of calculating "beta = 255 / min_scale_side" and first crop that region to make pyramid of crops out of that cropped region

I would be glad if you explain it bit more detail

Thanks in advance

there something wrong in the code

  z = video_id_z[rand_z].copy()    # use copy() here to avoid changing dictionary
    x = video_id_z[rand_x].copy()

    # read z and x，--------here the path of z actually is search image----，
    #I think it should be 
    # l=len(z['instance_path'])
    #img_z = cv2.imread(os.path.join(self.data_dir, (z['instance_path'])[:l-5]+'z.jpg'))
    #img_z = self.__cvt_color(img_z)
    img_z = cv2.imread(os.path.join(self.data_dir, z['instance_path']))
    img_z = self.__cvt_color(img_z)

    img_x = cv2.imread(os.path.join(self.data_dir, x['instance_path']))
    img_x = self.__cvt_color(img_x)