How to train and test with an image_size resolution of 256*256? about rvt

[imankgoyal]

Hi,

Thanks for the interest in our work. Yes you are right, changing img_size in mvt config might work (https://github.com/NVlabs/RVT/blob/master/rvt/mvt/config.py#L10). But one caveat is to accordingly adjust the patch_size (https://github.com/NVlabs/RVT/blob/master/rvt/mvt/config.py#L26) so that the img_size is divisible by it.

You might also try using a img_size of 253 so that it is divisible by the current patch_size of 11.

I am not sure if any other error because of corner cases like adjusting convolution size might creep in. Let me if you face any issue.

Best,
Ankit

[LemonWade]

Thank you for your answer. I will try it out as soon as possible. Also, I have recreated the dataset for the open_drawer task with 400 episodes from RLBench, and the images within the episodes are saved at a resolution of 224 pixels. I modified the IMAGE_SIZE in rvt.utils.peract_utils.py from the original 128 to 224 and successfully ran train.py. May I ask if this change won't have any significant impact on the experimental results? Additionally, if I train on a single task, can I reduce the number of epochs to around 2?

Thanks in advance！

[imankgoyal]

Oh I see. To clarify, there are two image sizes. One is the input image size (which you changed in rvt.utils.peract_utils.py). The other one is virtual image size (which I had referred to in my first respose in mvt_config).

I am unsure if changing the input image size would affect performance as in my experiments I assumed it to be fixed and same as PerAct. If I had to guess, it might not affect a lot on tasks like open drawer which need not be very precise. For other tasks it might affect performance.

I suppose you can reduce epochs (not a very clear name as it is same as steps) if you train on a single task. Ideally, I would suggest trying some set of epochs like 2, 4, 6 to make sure the training has converged.

[LemonWade]

Thank you very much for your answer! I'm currently trying to train on other tasks.