DofBot Sim2Real Error about omniisaacgymenvs-dofbotreacher

[dBlitz]

Hello,

Thanks for getting back to me.

1. Using Isaac Sim 2023.10-hotfix
2. Yes correct
3. Yes correct
4. Using jupyter notebook file in this repo
5. Yes correct same local network. IP for dofbot is 10.0.0.137

[dBlitz]

That was the problem I got it working! I guessed I was confused with the ReadMe.

Thank you!!

[j3soon]

Hi, nice to know that someone is trying out the Sim2Real feature!

I appreciate the detailed information. To summarize, the three videos demonstrates the following respectively:

1. The failure case where the sim2real flag is enabled.
2. The success case without enabling sim2real.
3. The realworld dofbot could be controlled (reset) by using the Jupyter Notebook.

---

I would like to confirm some additional information for ease of debugging:

1. Which Isaac Sim version are you using? I think you are using Isaac Sim 2023.1.0?
2. In video 1 / screenshot, which command did you use? I believe you are running the following command:

   python omniisaacgymenvs/scripts/rlgames_train.py task=DofbotReacher test=True num_envs=1 checkpoint=./runs/DofbotReacher/nn/DofbotReacher.pth

3. In video 2, are you using the pre-trained checkpoint or your own trained model when running the testing section (without enabling sim2real)?
4. In video 3, are you using the jupyter notebook file in this repo, or the one provided by dofbot?
5. Is the dofbot and your PC (with name eco) in the same local network? What's their respective IP?

[j3soon]

Hi,

As mentioned in the README, the Sim2Real code has not been tested on Isaac Sim 2023 yet. The error message seems to be a bug caused by the transition from Isaac Sim 2022 to Isaac Sim 2023. To confirm this issue, I'll need to test the code on my end. Unfortunately, I'm currently attending a conference abroad and don't have access to a physical Dofbot.

You could either wait for my return in 2 weeks to continue debugging the Isaac 2023 version, or consider trying out the Isaac Sim 2022 version in the isaac-sim-2022.1.1 branch. The Isaac Sim 2022 version has been tested thoroughly and I believe it should work well with your current setup.

[dBlitz]

I'm sure you're familiar with: https://github.com/NVIDIA-Omniverse/OmniIsaacGymEnvs/blob/main/docs/training_with_camera.md

As the next logical step. When do you think you'll get 2023 Isaac Sim working with Dofbot? I can then run with cameras and understand better current limitations and how I can potentially contribute.

[j3soon]

Hi,

I don't have access to a physical Dofbot now. However, I aim to begin investigating the issue within a few weeks.

I'll let you know when the issue is resolved.

Thanks!