How to setting the Multi-GPU for training? about tvretrieval

[jayleicn]

Hi @minjoong507,

This code does not support multi-GPU training, and normally there is also no need to use it as well since the whole training only takes 4 hours for a single 2080Ti, using full precision. If you want to use multi-GPUs, please refer to PyTorch's official tutorials.

Best,
Jie

[minjoong507]

Thank you for quick reply!

Is there any difference in performance between using multi-GPU and single-GPU?

Recently, I saw the issue about performance on HERO model. (link: linjieli222/HERO#14)

Thank you for your answering.

[jayleicn]

Hi @minjoong507,

Most likely yes, but it depends on how you implement it. For example, if hard negatives are mined across GPUs as in linjieli222/HERO#14, it is expected to see some improvement as better negatives are used (mined from a larger mini-batch). Meanwhile, as 3090 has a much larger memory than 2080Ti, you can increase the mini-batch size to get better negatives and thus some improvement.

Jie

[minjoong507]

I got it! Thanks :)