foolwood / siammask Goto Github PK
View Code? Open in Web Editor NEW[CVPR2019] Fast Online Object Tracking and Segmentation: A Unifying Approach
Home Page: http://www.robots.ox.ac.uk/~qwang/SiamMask
License: MIT License
[CVPR2019] Fast Online Object Tracking and Segmentation: A Unifying Approach
Home Page: http://www.robots.ox.ac.uk/~qwang/SiamMask
License: MIT License
I'm really interested in your amazing work and I've tried it ,here I would want to ask for your advice:
Are there any methods for scale variation? Because I found it not so effective to face the challenge,can you give some advice?.
The server runs the program. When running the demo, the following thread problems occur. How to solve them:
QObject::moveToThread: Current thread (0x2b3fd7fb8ff0) is not the object's thread (0x2b3fde28b410).
Cannot move to target thread (0x2b3fd7fb8ff0)
QPixmap: Must construct a QApplication before a QPaintDevice
Aborted (core dumped)
Hi, I'm wondering siammask's performance on OTB. I evaluated on OTB100 using siammask-3B bbox branch output with siammask_vot.pth with tracking parameters same as DaSiamRPN but only got 0.629/0.844 with AUC/Prec. Is this intrinsic or because of sensitivity to the hyper-paramerters?
Thanks for your time.
What are the data format to consider for training SiamMask? And what are the steps to follow to train it on this data?
Many thanks for your interesting work.
Hi Qiang,
After training siamrpn for about 200 iterations I spotted nan
in the loss. Do you think it is the problem of data preprocessing or somewhere in the code? Thank you.
Full training log attached below.
Best wishes,
Yiming
[2019-05-30 20:07:36,388-rk0-train_siamrpn.py#281] Epoch: [1][140/2343] lr: 0.001000 batch_time: 0.447263 (0.852683) data_time: 0.000222 (0.096190) rpn_cls_loss: 0.348680 (0.452594) rpn_loc_loss: 0.327530 (0.406526) siamrpn_loss: 0.676210 (0.859119)
[2019-05-30 20:07:36,389-rk0-log_helper.py# 97] Progress: 140 / 46860 [0%], Speed: 0.853 s/iter, ETA 0:11:03 (D:H:M)
[2019-05-30 20:07:41,014-rk0-train_siamrpn.py#281] Epoch: [1][150/2343] lr: 0.001000 batch_time: 0.428898 (0.826568) data_time: 0.000247 (0.089822) rpn_cls_loss: 0.330786 (0.445250) rpn_loc_loss: 0.339954 (0.400595) siamrpn_loss: 0.670740 (0.845844)
[2019-05-30 20:07:41,014-rk0-log_helper.py# 97] Progress: 150 / 46860 [0%], Speed: 0.827 s/iter, ETA 0:10:43 (D:H:M)
[2019-05-30 20:07:45,451-rk0-train_siamrpn.py#281] Epoch: [1][160/2343] lr: 0.001000 batch_time: 0.491518 (0.802561) data_time: 0.000388 (0.084235) rpn_cls_loss: 0.332730 (0.438211) rpn_loc_loss: 0.321817 (0.395307) siamrpn_loss: 0.654547 (0.833518)
[2019-05-30 20:07:45,451-rk0-log_helper.py# 97] Progress: 160 / 46860 [0%], Speed: 0.803 s/iter, ETA 0:10:24 (D:H:M)
[2019-05-30 20:07:50,009-rk0-train_siamrpn.py#281] Epoch: [1][170/2343] lr: 0.001000 batch_time: 0.449069 (0.782116) data_time: 0.000453 (0.079310) rpn_cls_loss: 0.321972 (0.431562) rpn_loc_loss: 0.312462 (0.390696) siamrpn_loss: 0.634434 (0.822258)
[2019-05-30 20:07:50,009-rk0-log_helper.py# 97] Progress: 170 / 46860 [0%], Speed: 0.782 s/iter, ETA 0:10:08 (D:H:M)
[2019-05-30 20:07:54,762-rk0-train_siamrpn.py#281] Epoch: [1][180/2343] lr: 0.001000 batch_time: 0.511367 (0.764989) data_time: 0.000424 (0.074935) rpn_cls_loss: 0.293845 (0.425333) rpn_loc_loss: 0.274334 (nan) siamrpn_loss: 0.568179 (nan)
[2019-05-30 20:07:54,763-rk0-log_helper.py# 97] Progress: 180 / 46860 [0%], Speed: 0.765 s/iter, ETA 0:09:55 (D:H:M)
[2019-05-30 20:07:59,350-rk0-train_siamrpn.py#281] Epoch: [1][190/2343] lr: 0.001000 batch_time: 0.401666 (0.748833) data_time: 0.000397 (0.071019) rpn_cls_loss: 0.301740 (0.419603) rpn_loc_loss: 0.295769 (nan) siamrpn_loss: 0.597508 (nan)
Hello, I run the project with only 10 fps on a titan xp, which is much slower than mentioned in README.md. Does anyone know how to accelerate? How to achieve a speed like real-time tracking?
Thanks for your great work! I have a difficulty to understand why and how to use COCO dataset to train the SiamMask?
Thanks
I run this code (我运行这个指令)
-->conda create -n siammask python=3.6
then i find this problem (发现如下问题)
Solving environment: failed
CondaHTTPError: HTTP 404 NOT FOUND for url https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/mian/noarch/repodata.json
Elapsed: 00:00.046715
The remote server could not find the noarch directory for the
requested channel with url: https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/mian
As of conda 4.3, a valid channel must contain a noarch/repodata.json
and
associated noarch/repodata.json.bz2
file, even if noarch/repodata.json
is
empty. please request that the channel administrator create
noarch/repodata.json
and associated noarch/repodata.json.bz2
files.
$ mkdir noarch
$ echo '{}' > noarch/repodata.json
$ bzip2 -k noarch/repodata.json
You will need to adjust your conda configuration to proceed.
Use conda config --show channels
to view your configuration's current state.
Further configuration help can be found at https://conda.io/docs/config.html.
If training with the refinement module, what is the loss for mask you use? a) Eq. 3 in the paper; b) Eq.3 plus something like binary cross entropy for the final mask; c)binary cross entropy for the final mask?
File "tools/demo.py", line 25, in
cfg = load_config(args)
File "/home/zz/SiamMask/utils/config_helper.py", line 11, in load_config
assert exists(args.config), '"{}" not exists'.format(args.config)
AssertionError: "config_davis.json" not exists
but there is config_davis.json in siammask,what should i do?Thanks for your reply!
Sir ,I'm very interesting in your thought about Siamese network.I want to run this code(SiamMask) in the win10.But there are some wrong in the VScode when it's running.The problem showed is that " ImportError
Building module utils.pyvotkit.region failed: ["TypeError: '>=' not supported between instances of 'NoneType' and 'str'\n"]".
And all package have been installed.It's running in VScode ,python 3.6.What should I do?I'm looking forward to your reply.
Thanks a lot.
So far, only get (0.421/0.598/0.234) based on SiamMask_VOT_LD.pth and still have large gap@Accuracy (0.598 vs. 0.615). Would like to know the possible root cause, thanks ~
Hello, anyone try this SiamMask project in ios? Can it is real-time and have a good result?
The evaluation code of VOT2016 may be set to low=108, high=371 and peak=168. I used the matlab vot-toolkit to get above data.
Can you share where I can access the 'resnet.model' weights, the pretrained resnet weights. And where the load_pretrain method is?
From line 37 of custom.py
(siammask) [liqiang@inspur siammask]$ bash test_mask_refine.sh config_vot.json SiamMask_VOT.pth VOT2016 0
[2019-03-14 19:42:16,619-rk0-test.py#551] Namespace(arch='Custom', config='config_vot.json', dataset='VOT2016', gt=False, log='log_test.txt', mask=True, refine=True, resume='SiamMask_VOT.pth', save_mask=False, visualization=False)
[2019-03-14 19:42:17,087-rk0-load_helper.py# 31] load pretrained model from SiamMask_VOT.pth
[2019-03-14 19:42:20,119-rk0-load_helper.py# 25] remove prefix 'module.'
[2019-03-14 19:42:20,121-rk0-load_helper.py# 18] used keys:356
[2019-03-14 19:42:20,184-rk0-test.py#595] Total Lost: 0
/export/userhome/liqiang/liqiang/Anaconda/ENTER/envs/siammask/lib/python3.6/site-packages/numpy/core/fromnumeric.py:2925: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/export/userhome/liqiang/liqiang/Anaconda/ENTER/envs/siammask/lib/python3.6/site-packages/numpy/core/_methods.py:85: RuntimeWarning: invalid value encountered in double_scalars
ret = ret.dtype.type(ret / rcount)
[2019-03-14 19:42:20,185-rk0-test.py#597] Mean Speed: nan FPS
when I run this file as what you write on readme.md, the result is like this
what's wrong with this error?
Before that, I read a paper on deep learning in target tracking called MDnet. MDnet seems to track only specific categories. I don't know if the same is true of Siamese, who is trained to track only certain categories of objects.
Could you share the training code?
It seems that in the Refine function, the target feature is not center placed in the cropped patch but why?
If I have multiple targets in the videos, could I track each of them using siamMask ?
Thanks !
This problem arises when testing vot. I hope you can help me with it. Thanks
run python ../../tools/demo.py --resume SiamMask_DAVIS.pth --config config_davis.json and get a error:
Traceback (most recent call last):
File "../tools/demo.py", line 7, in
from tools.test import *
ModuleNotFoundError: No module named 'tools'
Can someone help me solve this problem? Thank you.
@foolwood I am trying to understand the pipeline of the Siammask from the code. I found that you have instantiated multiple objects of "DepthCorr"-- separate for classification, regression and masking. Though in paper, I see the figure where correlation is shared by all the three head blocks.
Can you explain me if I am missing something or not understanding it properly?
As the title, sir.
python3 ../../tools/demo.py --resume SiamMask_DAVIS.pth --config config_davis.json
[2019-05-14 16:31:21,238-rk0-load_helper.py# 31] load pretrained model from SiamMask_DAVIS.pth
[2019-05-14 16:31:23,679-rk0-load_helper.py# 25] remove prefix 'module.'
[2019-05-14 16:31:23,680-rk0-load_helper.py# 18] used keys:356
Xlib: extension "RANDR" missing on display ":0".
Select a ROI and then press SPACE or ENTER button!
Cancel the selection process by pressing c button!
/home/sdu/.local/lib/python3.5/site-packages/torch/nn/functional.py:1890: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.
warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.")
Traceback (most recent call last):
File "../../tools/demo.py", line 54, in
state = siamese_track(state, im, mask_enable=True, refine_enable=True) # track
File "/home/sdu/SiamMask/tools/test.py", line 285, in siamese_track
_, contours, _ = cv2.findContours(target_mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
ValueError: not enough values to unpack (expected 3, got 2)
I haven't corrected the code in it, but after drawing the area and entering the "Enter" key, the above error occurred. Do you have a solution? Thank you very much.
when i download VOT2019, I got this issues, Why?
Hello, thanks for sharing your code. I encounted a problem when running 'demo.py'. I did all accoding to the Readme.
(siammask) bill@bill-QiTianM4550-N000:~/Documents/SiamMask/experiments/siammask$ python ../../tools/demo.py --resume SiamMask_DAVIS.pth --config config_davis.json
Traceback (most recent call last):
File "../../tools/demo.py", line 7, in
from tools.test import *
File "/home/bill/Documents/SiamMask/tools/test.py", line 30, in
from utils.pyvotkit.region import vot_overlap, vot_float2str
File "/home/bill/Documents/SiamMask/utils/pyvotkit/init.py", line 9, in
from . import region
ImportError: cannot import name 'region'
Then I deleted this project and reinstall it. However it shows again.
(siammask) [liqiang@inspur trackdat]$ VOT_YEAR=2016 bash scripts/download_vot.sh dl/vot2016
--2019-03-14 16:18:38-- http://data.votchallenge.net/vot2016/main/description.json
Resolving data.votchallenge.net (data.votchallenge.net)... 194.249.1.179
Connecting to data.votchallenge.net (data.votchallenge.net)|194.249.1.179|:80... connected.
HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable
The file is already fully retrieved; nothing to do.
scripts/download_vot.sh: line 19: jq: command not found
scripts/download_vot.sh: line 20: jq: command not found
scripts/download_vot.sh: line 21: jq: command not found
@foolwood where is this problem come from? anybody please help me to locate it
select multi ROI?
Hi,Qiang
Thks for your contribution!
I trained and tested the SiamRPN++ network you implemented and studied the code. Some questions now:
The output model structure (only layer3 in the custom version of resnet50, why?) is different from the SiamRPN++ model in pysot.
SiamRPN++ indicates in paper that the feature fusion of layer2_RPN, layer3_RPN and layer4_RPN in ResNet50 is not found in your code.
look forward to your reply.
This is really a great project
But can you open up its training code and training details, I want to know more about SiamMask, it is very helpful to me.
would you like share your three pretrained model on baiduyun or googledrive when you are free, thank you!!
D:\anaconda\envs\pytorch\python.exe D:/python_work/SiamMask-master/tools/demo.py
usage: demo.py [-h] --resume PATH [--config CONFIG] [--base_path BASE_PATH]
demo.py: error: the following arguments are required: --resume
There are some postprocessing methods? I can not obtain the results shown in the paper. The segmentation results use CRF?
我想用下 DAVIS model,但是想放在 pysot 下使用,请问该怎么转换?
Hello.
I met the following trouble when i running the demo.py, and I've made a lot of effort to solve it, but can't.
Traceback (most recent call last):
File "/home/aibc/Wen/Siamese/SiamMask/tools/demo.py", line 12, in <module>
from tools.test import siamese_track,siamese_init
ModuleNotFoundError: No module named 'tools'
Thanks for your help.
Traceback (most recent call last):
File "../../tools/demo.py", line 52, in
state = siamese_init(im, target_pos, target_sz, siammask, cfg['hp']) # init tracker
File "/home/lut/SiamMask/tools/test.py", line 157, in siamese_init
z_crop = get_subwindow_tracking(im, target_pos, p.exemplar_size, s_z, avg_chans)
File "/home/lut/SiamMask/tools/test.py", line 109, in get_subwindow_tracking
im_patch = cv2.resize(im_patch_original, (model_sz, model_sz))
cv2.error: OpenCV(3.4.3) /io/opencv/modules/imgproc/src/resize.cpp:4044: error: (-215:Assertion failed) !ssize.empty() in function 'resize'
Hello sir!
I want to use uav123 to evaluate it.But siammask use 8 parameters to describe the rectangle . But like Siam-RPN just use 4 . So if I want to evaluate the precision and success of it ,what conversion of rectangle should I do?
Thanks a lot for your reply!
flake8 testing of https://github.com/foolwood/SiamMask on Python 3.7.1
$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics
./utils/pysot/utils/statistics.py:57:21: F821 undefined name 'x'
overlaps = [x if u else 0 for u in unkown]
^
1 F821 undefined name 'x'
1
E901,E999,F821,F822,F823 are the "showstopper" flake8 issues that can halt the runtime with a SyntaxError, NameError, etc. These 5 are different from most other flake8 issues which are merely "style violations" -- useful for readability but they do not effect runtime safety.
name
name
in __all__
Did you also train SiamMask using random pairs of the same sequence from DAVIS dataset? I'm just curious because DAVIS has much much more occlusions than VOT. I doubt if siamese network works well on DAVIS. It's too difficult if the template is one part of the object and the target is another part of the object.
load pretrained model from SiamMask_DAVIS.pth
[2019-03-10 04:08:45,841-rk0-load_helper.py# 25] remove prefix 'module.'
[2019-03-10 04:08:45,843-rk0-load_helper.py# 18] used keys:356
: cannot connect to X server
I met the following trouble when i running the demo.py, and I've made a lot of effort to solve it, but can't.
ImportError: Building module region failed: ["distutils.errors.CompileError: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional\\VC\\Tools\\MSVC\\14.16.27023\\bin\\HostX86\\x64\\cl.exe' failed with exit status 2\n"]
Could anybody give me a hand? Thank you very much in advance!
Traceback (most recent call last):
File "../../tools/demo.py", line 56, in
state = siamese_track(state, im, mask_enable=True, refine_enable=True) # track
File "/home/zhouwenzhang/SiamMask-master/tools/test.py", line 254, in siamese_track
mask = net.track_refine((delta_y, delta_x)).cuda().sigmoid().squeeze().view(
File "/home/zhouwenzhang/SiamMask-master/experiments/siammask/custom.py", line 157, in track_refine
pred_mask = self.refine_model(self.feature, self.corr_feature, pos=pos)
File "/opt/conda/envs/pytorch-py3.6/lib/python3.6/site-packages/torch/nn/modules/module.py", line 468, in call
result = self.forward(*input, **kwargs)
File "/home/zhouwenzhang/SiamMask-master/experiments/siammask/custom.py", line 123, in forward
out = self.post0(F.upsample(self.h2(out) + self.v2(p2), size=(31, 31)))
File "/opt/conda/envs/pytorch-py3.6/lib/python3.6/site-packages/torch/nn/functional.py", line 1797, in upsample
return torch._C._nn.upsample_nearest2d(input, _scale_factor(2))
File "/opt/conda/envs/pytorch-py3.6/lib/python3.6/site-packages/torch/nn/functional.py", line 1769, in _scale_factor
'x'.join(map(str, input.size()))))
RuntimeError: output size specified in UpsamplingNearest (31x31) has to be divisible by the input size, but got: 1x32x15x15
Sorry,if I want to train this network, can i use only 1 dataset from Youtube-VOS, COCO, ImageNet-DET and ImageNet-VID, or all. thanks!
if my object out of my window,it always have bbox.Can you solve it??thanks
python ../../tools/demo.py --resume SiamMask_DAVIS.pth --config config_davis.json
Traceback (most recent call last):
File "../../tools/demo.py", line 20, in
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
AttributeError: 'module' object has no attribute 'device'
How can I solve the problem? Thank you very much.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.