zllrunning / siamesex.pytorch

A simplified PyTorch implementation of Siamese networks for tracking: SiamFC, SiamRPN, SiamRPN++, SiamVGG, SiamDW, SiamRPN-VGG.

SiameseX.PyTorch

A simplified PyTorch implementation of Siamese networks for tracking: SiamFC, SiamVGG, SiamDW, SiamRPN, SiamRPN++.

Warning: It is still in development, some files are just for the integrity of the project.

Dependencies

• python2.7
• pytorch == 0.4.0
• opencv
• mmcv

Currently supported models

• SiamFC
  • SiamVGG
  • SiamFCRes22
  • SiamFCIncep22
  • SiamFCNext22
• SiamRPN
  • SiamRPNVGG
  • SiamRPNRes22
  • SiamRPNIncep22
  • SiamRPNResNeXt22
• SiamRPN++

Backbones

• AlexNet
• VGG
• ResNet22
• Incep22
• ResNeXt22
• ResNet50

Demo

• Try SiamFC:

Clone this repo and run

python demo.py --model SiamFC

You can change --mdoel to other models like

python demo.py --model SiamFCNext22

• Try SiamRPN:

Download the pretrained model and put it in ./cp folder.

python demo_rpn.py --model SiamRPNResNeXt22

• Try SiamRPN++:

Download the pretrained model and put it in ./cp folder.

python demo_rpn.py --model SiamRPNPPRes50

• You'll see the following:

Training

• download VID dataset (I did not use Youtube-bb because of limited resources).
• download label files from here and run label_preprocess.py --file --output_file --path to replace my file path by yours, these three arguments should be treated carefully.
• train SiamFCNext22 tracker

python train.py --gpu 0 --config configs/SiamFC.py

• just replace --config argument and you can train other models

python train.py --gpu 0 --config configs/SiamRPN.py

TODO

We have accumulated the following to-do list, which we hope to complete in the near future

• Still to come:
  • Add testing code on common datasets
  • Add SiamRPN(AlexNet as backbone)
  • Add SiamRPN(VGG as backbone)
  • Add SiamRPN(ResNet, ResNext, Inception as backbone)

Citation

@inproceedings{bertinetto2016fully,
  title={Fully-convolutional siamese networks for object tracking},
  author={Bertinetto, Luca and Valmadre, Jack and Henriques, Joao F and Vedaldi, Andrea and Torr, Philip HS},
  booktitle={European conference on computer vision},
  pages={850--865},
  year={2016},
  organization={Springer}
}

@inproceedings{Li2019SiamVGGVT,
  title={SiamVGG: Visual Tracking using Deeper Siamese Networks},
  author={Yuhong Li and Xiaofan Zhang},
  year={2019}
}

@inproceedings{SiamDW_2019_CVPR,
    author={Zhang, Zhipeng and Peng, Houwen},
    title={Deeper and Wider Siamese Networks for Real-Time Visual Tracking},
    booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year = {2019}

@article{li2018siamrpn++,
  title={SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks},
  author={Li, Bo and Wu, Wei and Wang, Qiang and Zhang, Fangyi and Xing, Junliang and Yan, Junjie},
  journal={arXiv preprint arXiv:1812.11703},
  year={2018}
}
}

References

• SiamVGG
• siamfc-pytorch
• SiamDW
• pysot
• DaSiamRPN

siamesex.pytorch's Issues

How to determine self.final_score_sz？

Hi, thank you for sharing the codes!
Given x_sz=255 and z_sz=127, the self.final_score_sz is set as 273. But how is 273 calculated？How can I determine self.final_score_sz in SiamVGGTracker for my dataset？
Also, hyparameters such as tot_stride and response_up are not clear. Could you provide breif explanations about them?
Your help would be much appreciated.

can not find module named 'neck'

how to solve it?

when i excute the second command: python demo.py --model SiamFC
something is wrong like:

transfer learning strategy

Hi.
Can you please share some transfer learning strategy to start with
without training from scratch?
thank you

can you tell me the result of SiamDW in VOT2018 and VOT2016 (trained by COCO, ImageNet DET , ImageNet VID, and YouTube-BoundingBoxes Dataset)?

can you tell me the result of SiamDW in VOT2018 and VOT2016 (trained by COCO, ImageNet DET , ImageNet VID, and YouTube-BoundingBoxes Dataset)?

Any plans for the cfnet?

Thanks for your nice work! It helps me a lot.

Are there plans for the cfnet, the siamfc-v2? I notice that there is not a Tensorflow or Pytorch version thus making it helpful if one did it.

Besides, I read your to-do lists and guess you are working for testing trackers on common datasets. GOT-10k toolkit is a useful tool for this, just a small suggestion.

Can you make a file to describe the right way to set import path?

I run the command python demo_rpn.py --model SiamRPNResNeXt22 in the directory "SiameseX.PyTorch", I got the error "from neck import AdjustLayer, AdjustAllLayer". Then, I changed it to from .neck import AdjustLayer, AdjustAllLayer and rerun that command, I got an another error "ImportError: cannot import name 'get_subwindow_tracking'".
Through your code, I found these problems are results from the disordered import path management.
So, Can you make a description to make them clear?

Training on your own dataset

Can you please tell us the steps to train the models on our own dataset in the documentation?

Do you know the loss of siamrpn++ in detail?

Do you know the loss of siamrpn++ in detail?

ModuleNotFoundError: No module named 'neck'

Whenever I try to run the demo code, I receive the following error:

Traceback (most recent call last):
  File "demo.py", line 10, in <module>
    from demo_utils.siamvggtracker import SiamVGGTracker
  File "C:\Users\Harinder\Desktop\SiameseX.PyTorch\demo_utils\siamvggtracker.py", line 12, in <module>
    from demo_utils.siamese import SiameseNet
  File "C:\Users\Harinder\Desktop\SiameseX.PyTorch\demo_utils\siamese.py", line 16, in <module>
    import models.builder as builder
  File "C:\Users\Harinder\Desktop\SiameseX.PyTorch\models\builder.py", line 10, in <module>
    from neck import AdjustLayer, AdjustAllLayer
ModuleNotFoundError: No module named 'neck'

Can you provide trained models?

Multiple tracker instances

Can this implementation handle multiple tracker instances to track several objects on a frame.
The reason for asking: SiamRPN PYSOT implementation cannot handle it, since SOT requires initialization.

where is ilsvrc_vid.txt?

I download the ilsvrc_vid2015 Dataset， but i can't find ilsvrc_vid.txt can you help me?

Which code handles the sampling problem of random_shift_factor=32？

Which code handles the sampling problem of random_shift_factor=32？

about the performance

Have you tested the performance of SiamRPN?

how to use it for custom images/ videos?

Hi, can you provide any guidance in using this code for custom images/videos? what if i want to draw bounding box first and then track it? to make it more usable...thanks

Problem with pretrained weights

I've recently been attempting to run the demo_rpn.py script with the pretrained weights you have made available. Unfortunately, it appears that the weights don't match the model.

Placing a breakpoint at line 14 of demo_rpn_utils/utils.py and examining the model/loaded weights, I notice a disparity in the number of parameters:

For SiamRPNPPRes50
Weights in loaded dict: 458
Weights in state dict: 384

For SiamRPNResNeXt22
Weights in loaded dict: 180
Weights in state dict: 151

Any thoughts?

groundtruth

What does your groundtruth's mean,it has 8D vectors?