Why the 'DDP' mode cannot work on this framework about udat

[Jay-Ye]

Thank you for your comment.
Yes, exactly, we also found the uneven distribution of GPU memory during training. Though we haven't found the reason for now, it can be safely ignored since the training runs successfully.
And great thanks for your report of the bugs, we've fixed them accordingly.

[kongbia]

Thank you for your comment. Yes, exactly, we also found the uneven distribution of GPU memory during training. Though we haven't found the reason for now, it can be safely ignored since the training runs successfully. And great thanks for your report of the bugs, we've fixed them accordingly.

Hi, I tried to modify the line 136 of distributed.py
if param.requires_grad:
into
if param.requires_grad and param.grad is not None:
and it can trained with DDP successfully. The training time reduced from 10 hours to 4 hours in my machine.

However, I found the reason is that all parameters of ALIGN module have no grad (is None) but their requires_grad is True. I am quite confused about it.

[cjyiiiing]

Hi. Can you reproduce the results in the paper? For UDAT-CAR tested on NAT2021-test, I got 0.458(Success) and 0.655(Precision), but in the paper it's 0.483 and 0.687.

[Jay-Ye]

Hi. Can you reproduce the results in the paper? For UDAT-CAR tested on NAT2021-test, I got 0.458(Success) and 0.655(Precision), but in the paper it's 0.483 and 0.687.

Could you please post more details of your testing/training? So we can figure things out.

[cjyiiiing]

Hi. Can you reproduce the results in the paper? For UDAT-CAR tested on NAT2021-test, I got 0.458(Success) and 0.655(Precision), but in the paper it's 0.483 and 0.687.

Could you please post more details of your testing/training? So we can figure things out.

The setting I used during preprocessing/training/testing is python 3.7.11, pytorch 1.6.0, cudatoolkit 10.1.243. All training/testing parameters remain unchanged. So is it the problem of environment? Or do I need to post some more other details?

[Jay-Ye]

Hi. Can you reproduce the results in the paper? For UDAT-CAR tested on NAT2021-test, I got 0.458(Success) and 0.655(Precision), but in the paper it's 0.483 and 0.687.

Could you please post more details of your testing/training? So we can figure things out.

The setting I used during preprocessing/training/testing is python 3.7.11, pytorch 1.6.0, cudatoolkit 10.1.243. All training/testing parameters remain unchanged. So is it the problem of environment? Or do I need to post some more other details?

Have you checked about the model you were running? Some people have reproduced the results in the paper, the environmental difference should not cause a huge performance drop.

[cjyiiiing]

Hi. Can you reproduce the results in the paper? For UDAT-CAR tested on NAT2021-test, I got 0.458(Success) and 0.655(Precision), but in the paper it's 0.483 and 0.687.

Could you please post more details of your testing/training? So we can figure things out.

The setting I used during preprocessing/training/testing is python 3.7.11, pytorch 1.6.0, cudatoolkit 10.1.243. All training/testing parameters remain unchanged. So is it the problem of environment? Or do I need to post some more other details?

Have you checked about the model you were running? Some people have reproduced the results in the paper, the environmental difference should not cause a huge performance drop.

I have checked every steps including preprocessing and training. Everything seems ok, except that when I run gen_json.py of NAT dataset I skip sequence "0175bike1_3" bacause there is no "0175bike1_3_gt.txt" in "pseudo_anno/". I retrain the model, but the the performance is still low:

------------------------------------------------------------------------------------------------------
|                        Tracker name                         | Success | Norm Precision | Precision |
------------------------------------------------------------------------------------------------------
| UDATCAR_A6000_snapshot_wrandomcheckpoint_e19_0.39_0.04_0.37 |  0.457  |     0.000      |   0.652   |
------------------------------------------------------------------------------------------------------

[Jay-Ye]

Hi. Can you reproduce the results in the paper? For UDAT-CAR tested on NAT2021-test, I got 0.458(Success) and 0.655(Precision), but in the paper it's 0.483 and 0.687.

Could you please post more details of your testing/training? So we can figure things out.

The setting I used during preprocessing/training/testing is python 3.7.11, pytorch 1.6.0, cudatoolkit 10.1.243. All training/testing parameters remain unchanged. So is it the problem of environment? Or do I need to post some more other details?

Have you checked about the model you were running? Some people have reproduced the results in the paper, the environmental difference should not cause a huge performance drop.

I have checked every steps including preprocessing and training. Everything seems ok, except that when I run gen_json.py of NAT dataset I skip sequence "0175bike1_3" bacause there is no "0175bike1_3_gt.txt" in "pseudo_anno/". I retrain the model, but the the performance is still low:

------------------------------------------------------------------------------------------------------
|                        Tracker name                         | Success | Norm Precision | Precision |
------------------------------------------------------------------------------------------------------
| UDATCAR_A6000_snapshot_wrandomcheckpoint_e19_0.39_0.04_0.37 |  0.457  |     0.000      |   0.652   |
------------------------------------------------------------------------------------------------------

I suggest you test the model we released (both the original version and the UDAT version) on your platform and compare the results with the paper, to figure out how environmental difference influences the results.

[cjyiiiing]

Hi. Can you reproduce the results in the paper? For UDAT-CAR tested on NAT2021-test, I got 0.458(Success) and 0.655(Precision), but in the paper it's 0.483 and 0.687.

Could you please post more details of your testing/training? So we can figure things out.

The setting I used during preprocessing/training/testing is python 3.7.11, pytorch 1.6.0, cudatoolkit 10.1.243. All training/testing parameters remain unchanged. So is it the problem of environment? Or do I need to post some more other details?

Have you checked about the model you were running? Some people have reproduced the results in the paper, the environmental difference should not cause a huge performance drop.

I have checked every steps including preprocessing and training. Everything seems ok, except that when I run gen_json.py of NAT dataset I skip sequence "0175bike1_3" bacause there is no "0175bike1_3_gt.txt" in "pseudo_anno/". I retrain the model, but the the performance is still low:

------------------------------------------------------------------------------------------------------
|                        Tracker name                         | Success | Norm Precision | Precision |
------------------------------------------------------------------------------------------------------
| UDATCAR_A6000_snapshot_wrandomcheckpoint_e19_0.39_0.04_0.37 |  0.457  |     0.000      |   0.652   |
------------------------------------------------------------------------------------------------------

I suggest you test the model we released (both the original version and the UDAT version) on your platform and compare the results with the paper, to figure out how environmental difference influences the results.

I test the model of the original version.

SiamCAR:

• when python=3.7.11, pytorch=1.6.0, cudatoolkit=10.1.243, the result is 0.422(Success) and 0.633(Precision);
• when python=3.6.1, pytorch=1.2.0, cudatoolkit=10.0.130, the result is 0.450(Success) and 0.670(Precision);

SiamBAN:

• when python=3.7.11, pytorch=1.6.0, cudatoolkit=10.1.243, the result is 0.271(Success) and 0.441(Precision);
• when python=3.7.13, pytorch=1.3.1, cudatoolkit=10.1.243, the result is 0.327(Success) and 0.540(Precision);

It seems that the environment influences the results a lot. Can you tell me your environment setting?

[Jay-Ye]

Hi. Can you reproduce the results in the paper? For UDAT-CAR tested on NAT2021-test, I got 0.458(Success) and 0.655(Precision), but in the paper it's 0.483 and 0.687.

Could you please post more details of your testing/training? So we can figure things out.

The setting I used during preprocessing/training/testing is python 3.7.11, pytorch 1.6.0, cudatoolkit 10.1.243. All training/testing parameters remain unchanged. So is it the problem of environment? Or do I need to post some more other details?

Have you checked about the model you were running? Some people have reproduced the results in the paper, the environmental difference should not cause a huge performance drop.

I have checked every steps including preprocessing and training. Everything seems ok, except that when I run gen_json.py of NAT dataset I skip sequence "0175bike1_3" bacause there is no "0175bike1_3_gt.txt" in "pseudo_anno/". I retrain the model, but the the performance is still low:

------------------------------------------------------------------------------------------------------
|                        Tracker name                         | Success | Norm Precision | Precision |
------------------------------------------------------------------------------------------------------
| UDATCAR_A6000_snapshot_wrandomcheckpoint_e19_0.39_0.04_0.37 |  0.457  |     0.000      |   0.652   |
------------------------------------------------------------------------------------------------------

I suggest you test the model we released (both the original version and the UDAT version) on your platform and compare the results with the paper, to figure out how environmental difference influences the results.

I test the model of the original version.

SiamCAR:

• when python=3.7.11, pytorch=1.6.0, cudatoolkit=10.1.243, the result is 0.422(Success) and 0.633(Precision);
• when python=3.6.1, pytorch=1.2.0, cudatoolkit=10.0.130, the result is 0.450(Success) and 0.670(Precision);

SiamBAN:

• when python=3.7.11, pytorch=1.6.0, cudatoolkit=10.1.243, the result is 0.271(Success) and 0.441(Precision);
• when python=3.7.13, pytorch=1.3.1, cudatoolkit=10.1.243, the result is 0.327(Success) and 0.540(Precision);

It seems that the environment influences the results a lot. Can you tell me your environment setting?

Hey, the environment used in the original paper is:

PyTorch 1.11.0
CUDA11.6/cudnn8.4.0
Python 3.9.12