Some questions about the results on Hollywood2 and UCFsports about hd2s

[GemLuo]

@simopal6 @cspampin @feiyanhu @yuko95 @ belli13

[GemLuo]

@belli13

[belli13]

Hi @GemLuo ,
thank you for your interest in our work.

1. Yes, the results on Table 2 refers to HD2S trained on DHF1K train set and tested on Hollywood2 and UCFSports test sets. In Table 7 we basically report:

• Single-source : training and testing on same dataset only.
• Multi-source : training on DHF1K, Hollywhood2 and UCFSports train sets, and testing on a single dataset.
• Domain-specific : same as Multi-source with HD2S_DSL model.

2. As we stated in the paper, we report the results using only DHF1K validation set. However, during our experiments we tried with different configurations, for example using all the three val sets. The only change to do in train_DSL.py is adding/removing the items in validation_datasets list opportunely. We already updated the code to use only DHF1K as val set.
   In the meanwhile, we added splitTrainVal for Hollywood2.

Thank you.

[GemLuo]

Thanks for your help! But i still have other questions.
In train_DSL.py , there are some setting for HD2S_DSL.

But i think it should be
, if i want to build a HD2S_DSL model.

[belli13]

Exactly.

[GemLuo]

I am so glad to receive your reply, thank you so much!

[GemLuo]

Could you please show me the python environment dependency and all package's version? I want to know the code for metrics?

[GemLuo]

I got the predicted saliency map of DHF1K validation set by loading the HD2S weights that you given in this project. I also train the HD2S model by myself, but both of them get lower performance than the paper stated.

I use pytorch(1.0) and python 3.7.9, and the code for metrics from https://github.com/wenguanwang/DHF1K/blob/master/code_for_Metrics.zip
And would you mind giving me your contact information like wechat, whatsapp, telegram or other social software? It is not convenient to communicate by e-mail. Please!

[GemLuo]

There are a problem after I run the train_DSL.py.
/storage/LHY2/Dataset/VideoSaliency/DHF1K
Init Dataset...
Init Dataset Completed.
/storage/LHY2/Dataset/VideoSaliency/Hollywood2_actions
Init Dataset...
Init Dataset Completed.
/storage/LHY2/Dataset/VideoSaliency/ucf
Init Dataset...
Init Dataset Completed.
HD2S_DSL_training_demo_1
Loading weights...
Loading done!
/storage/LHY/anaconda/envs/pytorch/lib/python3.7/site-packages/torch/nn/functional.py:2539: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
Traceback (most recent call last):
File "train_DSL.py", line 481, in
main()
File "train_DSL.py", line 159, in main
summ = summary(model, input_size=(3, 16, 128, 192))
File "/storage/LHY/anaconda/envs/pytorch/lib/python3.7/site-packages/torchsummary/torchsummary.py", line 72, in summary
model(*x)
File "/storage/LHY/anaconda/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/storage/LHY2/Code/hd2s/models/HD2S_DSL.py", line 280, in forward
out = self.getattr(f'GL_{source_str}')(out)
File "/storage/LHY/anaconda/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/storage/LHY2/Code/hd2s/models/GaussianFilter/LearnableGaussianFilter.py", line 35, in forward
x = F.conv2d(x, kernel, padding=self.half_k)
RuntimeError: Input type (torch.cuda.IntTensor) and weight type (torch.cuda.FloatTensor) should be the same

I think the underlined code is the reason that I got this problem. So I want to know why you write this line, it seems like the output changed nothing except its type.

[GemLuo]

@belli13 @chica9 Could you please show me the python environment dependency and all package's version? I want to know the code for metrics. Please! I hope I can get your help.

[chica9]

Hi #@GemLuo,
we just generated the saliency prediction of the DHF1K validation set using the code in the repo and the provided weights. We also calculated the metrics with the the same matlab code that you have used, and that's our results:

We have no idea why you got those results. Also, please note that the matlab code uses a subset of only 50k frames, rather than the whole frames, to compute metrics, so slight differences in results can happen.

About the RuntimeError you got, we have ran train_DSL.py again and everything works fine.

RuntimeError: Input type (torch.cuda.IntTensor) and weight type (torch.cuda.FloatTensor) should be the same

I think the underlined code is the reason that I got this problem. So I want to know why you write this line, it seems like the output changed nothing except its type.

The type of output in the line you indicated doesn't change, but only the number of digits do.

This is the environment dependency.
environment.zip

It's better to keep communicating here to leave tracks of our discussion for potentially interested people.

[GemLuo]

If it is convenient, could you please send me the generated result of validation set of DHF1K? @chica9 @belli13