Cannot reproduce the performance of EmbCLIP on the ObjectNav task of the RoboTHOR about allenact

[Lucaweihs]

Hi @hutchinsonian, it's been quite a while since I looked at these details... Can you tell me what commit ID AI2-THOR starts when you're running the evaluation (you can find this by running ps aux | grep thor when running evaluation)? There have been some changes between AI2-THOR versions would could cause some differences in metrics,

[hutchinsonian]

Hi @hutchinsonian, it's been quite a while since I looked at these details... Can you tell me what commit ID AI2-THOR starts when you're running the evaluation (you can find this by running ps aux | grep thor when running evaluation)? There have been some changes between AI2-THOR versions would could cause some differences in metrics,

I got this:

xxx+ 3414782 29.8  0.1 8108624 761272 pts/13 Sl+  02:06   0:41 /home/xxx/.ai2thor/releases/thor-Linux64-bad5bc2b250615cb766ffb45d455c211329af17e/thor-Linux64-bad5bc2b250615cb766ffb45d455c211329af17e -screen-fullscreen 0 -screen-quality 7 -screen-width 400 -screen-height 300

And I eval this on the headless machine line, I have run sudo python scripts/startx.py & to start an x-display, the code looks to run without any problems(score of 0.17). Is there any reason why the performance of the model decreases when running commit id=bad5bc2b250615cb766ffb45d455c211329af17e version of AI2-THOR on a headless machine?

Do I need to follow the second point mentioned in the link and set the commit id to 91139c909576f3bf95a187c5b02c6fd455d06b48?
When I change the commit id to 91139c909576f3bf95a187c5b02c6fd455d06b48 and set THOR_IS_HEADLESS = True, I got this error:

ValueError: Invalid commit_id: 91139c909576f3bf95a187c5b02c6fd455d06b48 - no build exists for arch=Linux platforms=Linux64