Hello! I've been testing out a policy that I trained with your repository on the Go2 robot in real life and I've found that only crawl works really well. The good news is that the rest of the skills work in Isaac Gym (except tilt), but unfortunately they don't work in real life. I think the distillation might have been undertrained because I ran it on 32 num_envs instead of the 256 specified in the repository due to graphics card limitations. I've attached a graph of distillation steps 4 and 7 (purple and blue, respectively) below. I'd be really grateful if you could help me figure out what the distillation loss should be at the end of training for a robust launch in real life.

This code is apply to go1.Does this code is suit with go2.

    def _get_proprioception_obs(self):
  ......
        return torch.cat([
            torch.zeros((1, 3), device= self.model_device), # no linear velocity
            base_ang_vel * self.obs_scales["ang_vel"],
            projected_gravity,
            self.command_buf * self.commands_scale,
            (dof_pos - self.default_dof_pos) * self.obs_scales["dof_pos"],
            dof_vel * self.obs_scales["dof_vel"],
            self.actions
        ], dim= -1)

hello , mate , why you set linear velociy =0 by torch.zeros((1, 3), device= self.model_device), # no linear velocity , did you also train the RL model with obervation's linear velocity=0 in isaacgym ?

Thank you very much for sharing your resulting code.

I am hardware testing your results using a Unitree Go1 robot.

I only modified 1 line in your code.
I modified angle_tolerance=0.2 in line 289 of a1_ros_run.py to 2.0 and tested it.

There seem to be some issues.
Could you tell me where I need to change the code?

[Video-1]

1. After running python a1_ros_run.py --mode upboard --logdir Nov02..., the robot does not wake up immediately and starts shaking.
   And after a few seconds, the robot stands up.
2. And when the robot stands up, its upper body is tilted because it cannot straighten its hind legs.

[Video-2]

3. When I control the robot with the joystick, the robot becomes unstable and falls.

Could you tell me where the problem is?

Thank you.

Thank you for opening up your code.
My code suddenly terminates at a certain moment and displays "segment error (core has been dumped)", and the interruption time is also quite random. What is the problem? How should we solve it? Thanks~

Hello.
I am running python legged_gym/scripts/play.py --task a1_jump --load_run Apr06_01-05-16_Skills_jump_from_Apr05_10-43-00_WalkingBase _pEnergySubsteps2e-5_aScale0.5 ran into a similar problem to #11.
After reading your reply I tried to change the track_block_length in a1_jump_config.py under parkour/legged_gym/envs/a1 from 1.6 to 2 and n_obstacles_per_track to 3. In addition, I also changed the parkour/legged_gym/legged_gym/utils/terrain/barrier_track.py also to 2 and 3. But it doesn't work for me, it still reports the same error. Looking forward to your answer!

(corl) bld@bld-System-Product-Name:~/corl/parkour/legged_gym$ python legged_gym/scripts/play.py --task a1_jump --load_run Apr06_01-05-16_Skills_jump_from_Apr05_10-43-00_WalkingBase_pEnergySubsteps2e-5_aScale0.5
Importing module 'gym_38' (/home/bld/corl/isaacgym/python/isaacgym/_bindings/linux-x86_64/gym_38.so)
Setting GYM_USD_PLUG_INFO_PATH to /home/bld/corl/isaacgym/python/isaacgym/_bindings/linux-x86_64/usd/plugInfo.json
PyTorch version 1.13.0
Device count 1
/home/bld/corl/isaacgym/python/isaacgym/_bindings/src/gymtorch
Using /home/bld/.cache/torch_extensions/py38_cu116 as PyTorch extensions root...
Emitting ninja build file /home/bld/.cache/torch_extensions/py38_cu116/gymtorch/build.ninja...
Building extension module gymtorch...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module gymtorch...
Setting seed: 1
Using LeggedRobotField.__init__, num_obs and num_privileged_obs will be computed instead of assigned.
Not connected to PVD
+++ Using GPU PhysX
Physics Engine: PhysX
Physics Device: cuda:0
GPU Pipeline: enabled
Traceback (most recent call last):
  File "legged_gym/scripts/play.py", line 340, in <module>
    play(args)
  File "/home/bld/anaconda3/envs/corl/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "legged_gym/scripts/play.py", line 164, in play
    env, _ = task_registry.make_env(name=args.task, args=args, env_cfg=env_cfg)
  File "/home/bld/corl/parkour/legged_gym/legged_gym/utils/task_registry.py", line 99, in make_env
    env = task_class(   cfg=env_cfg,
  File "/home/bld/corl/parkour/legged_gym/legged_gym/envs/base/legged_robot_field.py", line 20, in __init__
    super().__init__(cfg, sim_params, physics_engine, sim_device, headless)
  File "/home/bld/corl/parkour/legged_gym/legged_gym/envs/base/legged_robot.py", line 72, in __init__
    super().__init__(self.cfg, sim_params, physics_engine, sim_device, headless)
  File "/home/bld/corl/parkour/legged_gym/legged_gym/envs/base/base_task.py", line 84, in __init__
    self.create_sim()
  File "/home/bld/corl/parkour/legged_gym/legged_gym/envs/base/legged_robot.py", line 238, in create_sim
    self._create_terrain()
  File "/home/bld/corl/parkour/legged_gym/legged_gym/envs/base/legged_robot_field.py", line 632, in _create_terrain
    self.terrain.add_terrain_to_sim(self.gym, self.sim, self.device)
  File "/home/bld/corl/parkour/legged_gym/legged_gym/utils/terrain/barrier_track.py", line 600, in add_terrain_to_sim
    self.build_heightfield_raw()
  File "/home/bld/corl/parkour/legged_gym/legged_gym/utils/terrain/barrier_track.py", line 444, in build_heightfield_raw
    heightfield_noise = TerrainPerlin.generate_fractal_noise_2d(
  File "/home/bld/corl/parkour/legged_gym/legged_gym/utils/terrain/perlin.py", line 69, in generate_fractal_noise_2d
    noise += amplitude * TerrainPerlin.generate_perlin_noise_2d((xSamples, ySamples), (xScale, yScale)) * zScale
  File "/home/bld/corl/parkour/legged_gym/legged_gym/utils/terrain/perlin.py", line 50, in generate_perlin_noise_2d
    n00 = np.sum(grid * g00, 2)
ValueError: operands could not be broadcast together with shapes (593,464,2) (592,464,2) 

Hello, thanks a lot for your work and detailed technical documentation! However, I'm trying to reproduce it and found that you've removed the "go1_climb" environment and added "go1_remote", "go1_jump " and "go1_down". May I ask what are the tasks represented by these three new environments? Looking forward to your answers!

Thanks for your excellent code framework!
I have a concern if I want to deploy the code on Go2. Can I follow the instructions on the simulation? And what is the difference in deployment between Go1 and Go2? Is it just the SDK?
Thank you very much!

After first build, I was able to run roslaunch unitree_legged_real robot.launch, but after restart, I keep getting udp_recv error: -1 message translate might lost once and unable to get the info from robot any more.

Rebuild didn't solve the problem. The sdk version is 3.8.6, and I'm pretty sure all packages were built successfully.

When I try to catkin build the related project-unitree_ros_real-go1 branch from Tsinghua-MARS-Lab, it rises an error 'cannot find -lunitree_legged_sdk'

...
Starting  >>> unitree_legged_real                                              
_______________________________________________________________________________
Errors     << unitree_legged_real:make /home/pdj/parkour_ros_real/logs/unitree_legged_real/build.make.001.log
/usr/bin/ld: cannot find -lunitree_legged_sdk
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/unitree_ros.dir/build.make:107: /home/pdj/parkour_ros_real/devel/.private/unitree_legged_real/lib/libunitree_ros.so] Error 1
make[1]: *** [CMakeFiles/Makefile2:1330: CMakeFiles/unitree_ros.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:141: all] Error 2
cd /home/pdj/parkour_ros_real/build/unitree_legged_real; catkin build --get-env unitree_legged_real | catkin env -si  /usr/bin/make --jobserver-auth=3,4; cd -

...............................................................................
Failed     << unitree_legged_real:make           [ Exited with code 2 ]        
Failed    <<< unitree_legged_real                [ 0.3 seconds ]    
...           

I have coupied the unitree_legged_sdk to the src directory. How can I solve this problem? Thanks!

Hi,

The code uses self.contact_forces to compute the reward. It is initialized here. But I cannot find where it is updated after each simulation step. Can you please point me to the relevant part of the code?

When I tried to run the command in T_visual and T_gru, they raised Illegal instruction (core dumped) and ended.

(parkour_venv) unitree@nx:~/parkour_deploy$ python go1_visual_embedding.py --logdir Nov02_16-18-16_674k_distill_crawljumpjumpleap_vDelay0.25-0.30_camPitch0.52_depthMax2.0_jumpNov02_11-07-49_leapVel1.5_jumpOffset0.05_oracleResetWhenSkill_fricMax2.0_noTanh_fromOct28_16-16-58
Illegal instruction (core dumped)
(parkour_venv) unitree@nx:~/parkour_deploy$ python a1_ros_run.py --mode upboard --logdir Nov02_16-18-16_674k_distill_crawljumpjumpleap_vDelay0.25-0.30_camPitch0.52_depthMax2.0_jumpNov02_11-07-49_leapVel1.5_jumpOffset0.05_oracleResetWhenSkill_fricMax2.0_noTanh_fromOct28_16-16-58
Illegal instruction (core dumped)
(parkour_venv) unitree@nx:~/parkour_deploy$ 

The unitree_legged_real command went well. I am presently running in the unitree@nx. I also tried with unitree@unitree-desktop, the same problem occured.

What's the problem and how to deal with it? Thanks!

Hi, thank you for your great job about parkour. I encountered some problems when I ran "barrier_track.py". "IndexError: tensors used as indices must be long, int, byte or bool tensors." What I need to do?

Error reported after running the training program:

AttributeError: module 'torch' has no attribute 'nanmean'

Thanks a lot for providing such wonderful work and clean codes! I'm curious about the results of RND and Oracles w/o Soft Dyn in Fig.7 and Fig. 9 in the paper (https://robot-parkour.github.io/resources/Robot_Parkour_Learning.pdf). In my opinion, they both directly train in the hard dynamics constraints by PPO and PPO+RND respectively, is that right? Also, Where is the code about RND in this repo?

Thanks again!

I trained the go1_feild script directly and got 0 mean reward from beginning to the end. This happen when training the a1_feild script. Then follow the experience of a1_feild, I change only_positive_rewards = True to only_positive_rewards = Flase. However, the mean reward raise only to aound -1.6 in the end. The image below shows the situation.

Then I played the policy. The go1 in 0 reward case shows a method of stuck in strange pose and almost without any motion. The go1 in -1.6 reward case keeps hitting the ground and resetting. The image below shows the -1.6 reward case.

I wonder why this happens? What shall I do to overcome this? Thanks!

Thanks a lot for providing such wonderful work and clean codes! I'm curious about the results of Go1.

For running A1 in the simulation via PPO, almost all tasks (field, crawl, jump, leap, tilt) can get rewards of ~20. However, when I run PPO for training Go1, the training is not ideal, some tasks get always 0 and some tasks get negative returns.

Are there any specific parameters that need to be set for Go1? What is the expected cumulative reward for Go1 tasks?

Thanks again!

Thank you for open-sourcing the code; it's impressive work! I'm working on deploying the network to Go1 EDU using your ros_real implementation. Could you provide details on setting up ROS and other dependencies on the Jetson platform? Are there any specific instructions or documentation for this process?

Thanks to your excellent work!
I would like to know if camera calibration is required. If not, please tell me why.

Hi, I am trying to reproduce your great control algorithm, but I encountered a problem when I ran legged gym using this command: 'python legged_gym/scripts/train.py --headless --task a1_field'

And then I got this error:

python legged_gym/scripts/train.py --headless --task a1_field
Importing module 'gym_38' (/home/user/workspace/parkour/isaacgym/python/isaacgym/_bindings/linux-x86_64/gym_38.so)
Setting GYM_USD_PLUG_INFO_PATH to /home/suntao/workspace/parkour/isaacgym/python/isaacgym/_bindings/linux-x86_64/usd/plugInfo.json
PyTorch version 1.10.0+cu113
Device count 1
/home/user/workspace/parkour/isaacgym/python/isaacgym/_bindings/src/gymtorch
Using /home/user/.cache/torch_extensions/py38_cu113 as PyTorch extensions root...
Emitting ninja build file /home/user/.cache/torch_extensions/py38_cu113/gymtorch/build.ninja...
Building extension module gymtorch...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module gymtorch...
Setting seed: 1
Using LeggedRobotField.init, num_obs and num_privileged_obs will be computed instead of assigned.
Not connected to PVD
+++ Using GPU PhysX
Physics Engine: PhysX
Physics Device: cuda:0
GPU Pipeline: enabled
Traceback (most recent call last):
File "legged_gym/scripts/train.py", line 49, in
train(args)
File "legged_gym/scripts/train.py", line 43, in train
env, env_cfg = task_registry.make_env(name=args.task, args=args)
File "/home/suntao/workspace/parkour/legged_gym/legged_gym/utils/task_registry.py", line 99, in make_env
env = task_class( cfg=env_cfg,
File "/home/suntao/workspace/parkour/legged_gym/legged_gym/envs/base/legged_robot_field.py", line 19, in init
super().init(cfg, sim_params, physics_engine, sim_device, headless)
File "/home/suntao/workspace/parkour/legged_gym/legged_gym/envs/base/legged_robot.py", line 72, in init
super().init(self.cfg, sim_params, physics_engine, sim_device, headless)
File "/home/suntao/workspace/parkour/legged_gym/legged_gym/envs/base/base_task.py", line 84, in init
self.create_sim()
File "/home/suntao/workspace/parkour/legged_gym/legged_gym/envs/base/legged_robot.py", line 238, in create_sim
self._create_envs()
File "/home/suntao/workspace/parkour/legged_gym/legged_gym/envs/base/legged_robot_field.py", line 561, in _create_envs
self.motor_strength = torch_rand_float(
RuntimeError: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript (most recent call last):
RuntimeError: nvrtc: error: invalid value for --gpu-architecture (-arch)

I thought this may related to the compatibility of the GPU, IsaacGym, and GPU driver..
So I am curious which Nvidia GPU you used.

Thank you!

Whether the initialization of the student network uses the parameters of the teacher network at the begining of distillation

When I'm trying to install rsl_rl, the requirement for torchvision is >=0.5.0, how did you install torchvision on the board?

Hi,
When I set selected = "TerrainPerlin" to train a walk policy, an error occurred：
File "/home/robot/parkour-main/legged_gym/legged_gym/utils/terrain/perlin.py", line 20, in __init__ assert(self.xSize == cfg.horizontal_scale * self.tot_rows and self.ySize == cfg.horizontal_scale * self.tot_cols) AssertionError.
and when I tried to solve it by adding:
self.xSize = cfg.horizontal_scale * self.tot_rows self.ySize = cfg.horizontal_scale * self.tot_cols
before the 'assert' line in perlin.py, the error disappear, but the traing process stuck at the beginning.
Would you please solve this problem?

Thanks.

Hi,

I tried to install on a new conda env by following the installation instructions.

The problem is that when I try to run the example script
python legged_gym/scripts/train.py --headless --task a1_field

I get that the numpy version (1.24.5) is calling deprecated functions within the IsaacGym Preview:
Traceback (most recent call last): File "legged_gym/legged_gym/scripts/train.py", line 36, in <module> from legged_gym.envs import * File "/home/tomjur/legged_gym/legged_gym/envs/__init__.py", line 33, in <module> from .base.legged_robot import LeggedRobot File "/home/tomjur/legged_gym/legged_gym/envs/base/legged_robot.py", line 37, in <module> from isaacgym.torch_utils import * File "/home/tomjur/IsaacGym_Preview_4_Package/isaacgym/python/isaacgym/torch_utils.py", line 135, in <module> def get_axis_params(value, axis_idx, x_value=0., dtype=np.float, n_dims=3): File "/home/tomjur/miniconda3/envs/parkour/lib/python3.8/site-packages/numpy/__init__.py", line 305, in __getattr__ raise AttributeError(__former_attrs__[attr]) AttributeError: module 'numpy' has no attribute 'float'. np.floatwas a deprecated alias for the builtinfloat. To avoid this error in existing code, use floatby itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, usenp.float64 here. The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

Since the code is within Isaac, I can only solve the issue by changing the numpy version to pre-deprecation (numpy 1.19), but this causes the following error:

Traceback (most recent call last): File "legged_gym/legged_gym/scripts/train.py", line 36, in <module> from legged_gym.envs import * File "/home/tomjur/legged_gym/legged_gym/envs/__init__.py", line 33, in <module> from .base.legged_robot import LeggedRobot File "/home/tomjur/legged_gym/legged_gym/envs/base/legged_robot.py", line 38, in <module> from isaacgym import gymtorch, gymapi, gymutil File "/home/tomjur/IsaacGym_Preview_4_Package/isaacgym/python/isaacgym/gymtorch.py", line 109, in <module> _import_gymtorch() File "/home/tomjur/IsaacGym_Preview_4_Package/isaacgym/python/isaacgym/gymtorch.py", line 45, in _import_gymtorch gt = torch.utils.cpp_extension.load(name="gymtorch", sources=sources, extra_cflags=cflags, verbose=True) File "/home/tomjur/miniconda3/envs/parkour/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1124, in load return _jit_compile( File "/home/tomjur/miniconda3/envs/parkour/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1337, in _jit_compile _write_ninja_file_and_build_library( File "/home/tomjur/miniconda3/envs/parkour/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1423, in _write_ninja_file_and_build_library check_compiler_abi_compatibility(compiler) File "/home/tomjur/miniconda3/envs/parkour/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 294, in check_compiler_abi_compatibility if not check_compiler_ok_for_platform(compiler): File "/home/tomjur/miniconda3/envs/parkour/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 254, in check_compiler_ok_for_platform which = subprocess.check_output(['which', compiler], stderr=subprocess.STDOUT) File "/home/tomjur/miniconda3/envs/parkour/lib/python3.8/subprocess.py", line 415, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/home/tomjur/miniconda3/envs/parkour/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['which', 'c++']' returned non-zero exit status 1.

Would it be possible for you to publish your pip list? or maybe an updated install script in case the above is out of date?

Thanks,
Tom

May i ask why you deploy this model with python rather than c++?
It seems like deploying with c++ is more suitable for realtime control.

The reward of the trained policy a1_field is always 0 from the begining to the end.

...
################################################################################
                     Learning iteration 9999/10000                      

                       Computation: 16523 steps/s (collection: 1.199s, learning 0.288s)
               Value function loss: 0.0071
                    Surrogate loss: -0.0000
             Mean action noise std: 706.21
                       Mean reward: 0.00
               Mean episode length: 16.04
            Mean episode rew_alive: 0.0311
      Mean episode rew_frame_alive: 0.0400
Mean episode rew_exceed_dof_pos_limits: -0.0185
Mean episode rew_frame_exceed_dof_pos_limits: -0.0238
Mean episode rew_exceed_torque_limits_i: -0.0031
Mean episode rew_frame_exceed_torque_limits_i: -0.0040
Mean episode rew_legs_energy_substeps: -0.2344
Mean episode rew_frame_legs_energy_substeps: -0.3077
 Mean episode rew_tracking_ang_vel: 0.0002
Mean episode rew_frame_tracking_ang_vel: 0.0002
 Mean episode rew_world_vel_l2norm: -0.0108
Mean episode rew_frame_world_vel_l2norm: -0.0138
        Mean episode timeout_ratio: 0.0000
            Mean episode max_pos_x: 0.7908
            Mean episode min_pos_x: -0.1488
            Mean episode max_pos_y: 0.6007
            Mean episode min_pos_y: -0.5227
    Mean episode n_obstacle_passed: 0.0000
                Mean episode pos_x: 0.3871
Mean episode max_power_throughout_episode: 7562.7012
          Mean episode max_torques: 51.0595
Mean episode torque_exceed_count_substeps_per_envstep: 3.3441
Mean episode torque_exceed_count_envstep: 15.5487
Mean episode out_of_dof_pos_limit_count_substep: 30.7613
--------------------------------------------------------------------------------
                   Total timesteps: 245760000
                    Iteration time: 1.49s
                        Total time: 15273.20s
                               ETA: 1.5s

The related command is
.../parkour/legged_gym$ python legged_gym/scripts/train.py --headless --task a1_field
Is there anything I missed?

Despite the 0 reward, then I tried to play the result, it gave a KeyError: 'climb' error as shown below:

.../parkour/legged_gym$ python legged_gym/scripts/play.py --task a1_field --load_run Nov17_17-20-05_WalkingBase_pEnergySubsteps2e-5_aScale0.5
Importing module 'gym_38' (/home/pdj/parkour_env/isaacgym/python/isaacgym/_bindings/linux-x86_64/gym_38.so)
Setting GYM_USD_PLUG_INFO_PATH to /home/pdj/parkour_env/isaacgym/python/isaacgym/_bindings/linux-x86_64/usd/plugInfo.json
PyTorch version 1.10.0+cu113
Device count 1
/home/pdj/parkour_env/isaacgym/python/isaacgym/_bindings/src/gymtorch
Using /home/pdj/.cache/torch_extensions/py38_cu113 as PyTorch extensions root...
Emitting ninja build file /home/pdj/.cache/torch_extensions/py38_cu113/gymtorch/build.ninja...
Building extension module gymtorch...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module gymtorch...
Setting seed: 1
Using LeggedRobotField.__init__, num_obs and num_privileged_obs will be computed instead of assigned.
Not connected to PVD
+++ Using GPU PhysX
Physics Engine: PhysX
Physics Device: cuda:0
GPU Pipeline: enabled
Traceback (most recent call last):
  File "legged_gym/scripts/play.py", line 340, in <module>
    play(args)
  File "/home/pdj/anaconda3/envs/parkour_env38/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "legged_gym/scripts/play.py", line 164, in play
    env, _ = task_registry.make_env(name=args.task, args=args, env_cfg=env_cfg)
  File "/home/pdj/parkour_env/parkour/legged_gym/legged_gym/utils/task_registry.py", line 99, in make_env
    env = task_class(   cfg=env_cfg,
  File "/home/pdj/parkour_env/parkour/legged_gym/legged_gym/envs/base/legged_robot_field.py", line 20, in __init__
    super().__init__(cfg, sim_params, physics_engine, sim_device, headless)
  File "/home/pdj/parkour_env/parkour/legged_gym/legged_gym/envs/base/legged_robot.py", line 72, in __init__
    super().__init__(self.cfg, sim_params, physics_engine, sim_device, headless)
  File "/home/pdj/parkour_env/parkour/legged_gym/legged_gym/envs/base/base_task.py", line 84, in __init__
    self.create_sim()
  File "/home/pdj/parkour_env/parkour/legged_gym/legged_gym/envs/base/legged_robot.py", line 238, in create_sim
    self._create_terrain()
  File "/home/pdj/parkour_env/parkour/legged_gym/legged_gym/envs/base/legged_robot_field.py", line 632, in _create_terrain
    self.terrain.add_terrain_to_sim(self.gym, self.sim, self.device)
  File "/home/pdj/parkour_env/parkour/legged_gym/legged_gym/utils/terrain/barrier_track.py", line 614, in add_terrain_to_sim
    starting_height_px = self.add_track_to_sim(
  File "/home/pdj/parkour_env/parkour/legged_gym/legged_gym/utils/terrain/barrier_track.py", line 555, in add_track_to_sim
    obstacle_id = self.track_options_id_dict[obstacle_name]
KeyError: 'climb'

What should I do to solve these problems?

After running the a1_jump-virtual train, I tried to play the result. However, I got a error saying that operands could not be broadcast together with shapes (593,464,2) (592,464,2) . The details are as follows:

.../parkour/legged_gym$ python legged_gym/scripts/play.py --task a1_jump --load_run Nov26_17-29-57_Skills_jump_virtual_fromNov24_09-41-36
Importing module 'gym_38' (/home/pdj/parkour_env/isaacgym/python/isaacgym/_bindings/linux-x86_64/gym_38.so)
Setting GYM_USD_PLUG_INFO_PATH to /home/pdj/parkour_env/isaacgym/python/isaacgym/_bindings/linux-x86_64/usd/plugInfo.json
PyTorch version 1.10.0+cu113
Device count 1
/home/pdj/parkour_env/isaacgym/python/isaacgym/_bindings/src/gymtorch
Using /home/pdj/.cache/torch_extensions/py38_cu113 as PyTorch extensions root...
Emitting ninja build file /home/pdj/.cache/torch_extensions/py38_cu113/gymtorch/build.ninja...
Building extension module gymtorch...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module gymtorch...
Setting seed: 1
Using LeggedRobotField.__init__, num_obs and num_privileged_obs will be computed instead of assigned.
Not connected to PVD
+++ Using GPU PhysX
Physics Engine: PhysX
Physics Device: cuda:0
GPU Pipeline: enabled
Traceback (most recent call last):
  File "legged_gym/scripts/play.py", line 341, in <module>
    play(args)
  File "/home/pdj/anaconda3/envs/parkour_env38/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "legged_gym/scripts/play.py", line 165, in play
    env, _ = task_registry.make_env(name=args.task, args=args, env_cfg=env_cfg)
  File "/home/pdj/parkour_env/parkour/legged_gym/legged_gym/utils/task_registry.py", line 116, in make_env
    env = task_class(   cfg=env_cfg,
  File "/home/pdj/parkour_env/parkour/legged_gym/legged_gym/envs/base/legged_robot_field.py", line 20, in __init__
    super().__init__(cfg, sim_params, physics_engine, sim_device, headless)
  File "/home/pdj/parkour_env/parkour/legged_gym/legged_gym/envs/base/legged_robot.py", line 72, in __init__
    super().__init__(self.cfg, sim_params, physics_engine, sim_device, headless)
  File "/home/pdj/parkour_env/parkour/legged_gym/legged_gym/envs/base/base_task.py", line 84, in __init__
    self.create_sim()
  File "/home/pdj/parkour_env/parkour/legged_gym/legged_gym/envs/base/legged_robot.py", line 265, in create_sim
    self._create_terrain()
  File "/home/pdj/parkour_env/parkour/legged_gym/legged_gym/envs/base/legged_robot_field.py", line 651, in _create_terrain
    self.terrain.add_terrain_to_sim(self.gym, self.sim, self.device)
  File "/home/pdj/parkour_env/parkour/legged_gym/legged_gym/utils/terrain/barrier_track.py", line 600, in add_terrain_to_sim
    self.build_heightfield_raw()
  File "/home/pdj/parkour_env/parkour/legged_gym/legged_gym/utils/terrain/barrier_track.py", line 444, in build_heightfield_raw
    heightfield_noise = TerrainPerlin.generate_fractal_noise_2d(
  File "/home/pdj/parkour_env/parkour/legged_gym/legged_gym/utils/terrain/perlin.py", line 68, in generate_fractal_noise_2d
    noise += amplitude * TerrainPerlin.generate_perlin_noise_2d((xSamples, ySamples), (xScale, yScale)) * zScale
  File "/home/pdj/parkour_env/parkour/legged_gym/legged_gym/utils/terrain/perlin.py", line 49, in generate_perlin_noise_2d
    n00 = np.sum(grid * g00, 2)
ValueError: operands could not be broadcast together with shapes (593,464,2) (592,464,2) 

I also tried other skills like leap, crawl, tilt. The same problem happened with crawl while leap and tilt went well.

What should I do to fix it? Thanks!

While installing numpy and numpy ros i am following error :

numpy-ros-version depends on numpy >1.19 and numpy<1.20

Thanks a lot for providing such wonderful work and clean codes! I'm curious about the results of the success rate reported in the paper. Are these success rates evaluated in the real world? Or can we evaluate the success rate in the simulator when training a single skill (like jump, or leap)? It seems that the running code only reports the defined rewards.

Thanks again!

Hello, have you ever meet the problem with position protection of the leg 2 calf when run the node "roslaunch unitree_legged_real robot.launch" in the terminal. At the same time I also can hear the vibration sounds from all the joint motors.
Thank you very much!

Great work and thanks for your opensource!

I want to test the onboard code on A1 and downloaded unitree_legged_sdk v3.3.1. The problem is as long as i roslaunch unitree_legged_real robot.launch， the warn comes out [ WARN] [1701523939.918416399]: Cmd lost, position control as motor damping. DO NOT use this as a normal operation.

Here is the complete info in the terminal:

... logging to /home/ustc/.ros/log/209d6ec8-904e-11ee-9d14-0d79cc269739/roslaunch-dell5820-1436634.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://127.0.0.1:42471/

SUMMARY

PARAMETERS

• /a112138/R2_press_protect: True
• /a112138/cmd_check: True
• /a112138/cmd_lost_timelimit: 0.05
• /a112138/ctrl_level: low
• /a112138/dryrun: True
• /a112138/freeze_lost: False
• /a112138/pitch_protect_limit: 1.3
• /a112138/position_protect_limit: 0.0
• /a112138/power_protect_level: 8
• /a112138/publish_imu: False
• /a112138/publish_joint_state: False
• /a112138/publish_wirelessRemote: False
• /a112138/roll_protect_limit: 0.9
• /a112138/start_stand: False
• /a112138/torque_protect_limit: 38.0
• /a112138/udp_duration: 0.01
• /rosdistro: noetic
• /rosversion: 1.16.0

NODES
/
a112138 (unitree_legged_real/real_robot)

ROS_MASTER_URI=http://127.0.0.1:11311

process[a112138-1]: started with pid [1436675]
UDP Initialized. Port: 8080
[ INFO] [1701523939.860609570]: Motor will be initialized to mode 0, please put the robot on the ground or hang up.
[ INFO] [1701523939.861923790]: Udp initialized
[Loop Start] named: udp_recv, period: 9(ms), run at cpu: 3
[Loop Start] named: udp_send, period: 9(ms), run at cpu: 3
[Loop Start] named: udp_translate, period: 9(ms), cpu unspecified
[ INFO] [1701523939.868134807]: RosUdpHandler constructed and started
[ WARN] [1701523939.918416399]: Cmd lost, position control as motor damping. DO NOT use this as a normal operation.
[ WARN] [1701523969.968446309]: Cmd lost, position control as motor damping. DO NOT use this as a normal operation.

Looking forward to your reply!

Where is the a1_climb_config.py? I cannot find it in the current repo.

Thanks for your excellent work. I am facing segmentation fault while running python legged_gym/scripts/collect.py --headless --task a1_distill. I traced back for the problem, and found that it was caused by the forward_depth observation when the code is creating for the environment. Do you have any idea about it.

My GPU is A100-40G, pytorch version is 1.10.0+cu113.

Hello, may I ask, is deploying the trained code on A1 similar to the Go1 hardware you released? What do I need to be aware of if I want to port trained code to A1? I just contact with the physical robot, not familiar with, hope to get your reply! Thank you very much

When I used the go1 for Sim2Real, the error shows "udp_recv error: -3, message translate might lost once". Is the problem caused by the wrong SDK version? The SDK 3.3.1 is not support for go1.

Hello, thank you for your excellent open source work!

I noticed in the video that the robot keeps re-tryinng as it climbs. I would be grateful if you could tell me how you achieved this behaviour?
I'm now trying to train the Go2 robot using your repository. If the robot fails to climb the first time, it just stands in front of the obstacle and does not try again.

Hello, I am recently discovering on your robot parkour learning, trying to implement it on my computer.
I am following the legged_gym/README.md tutorial and after I followed the instructions on the Installation part,
I tried to implement the python legged_gym/scripts/train.py --headless --task a1_field command, found out that

I solved this by altering the corresponding file at isaacgym downloaded, reimplemented the command and found that

I installed tensorboard with pip and found that

I looked up docs on disutils and found out that this might be the issue on its version, but pip install disutils suggested that
ERROR: Could not find a version that satisfies the requirement distutils (from versions: none)
I thought this would be some issue on the version of other packages such as tensorboard, numpy or else, but I have no idea to what extent should I downgrade them in order to solve this disutils problem.
Do you have some ideas about this? Thanks a lot!

I found the go1 was unable to stand up directly from crouching state(L1+L2+Start, this command closes unitree's controller into normal mode) with the provided policy.
The T_ros, T_visul went well while the T_gru gave something like the follows:

...
Actor RNN: Memory(
  (rnn): GRU(176, 256)
)
Critic RNN: Memory(
  (rnn): GRU(81, 256)
)
[INFO] [1702284120.606872]: duration: 0.02, motor Kp: [40.0, 40.0, 40.0, 40.0, 40.0, 40.0, 40.0, 40.0, 40.0, 40.0, 40.0, 40.0], motor Kd: [0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5]
[INFO] [1702284120.622811]: Using stand only model, please make sure the proprioception is 48 dim.
[INFO] [1702284120.630576]: Using stand only model, -36 to -24 must be joint position.
[INFO] [1702284120.743915]: UnitreeA1Real.low_state_buffer acquired, stop waiting.
[INFO] [1702284120.752136]: Robot standing up, please wait ...
[INFO] [1702284120.881907]: a1_ros_run recieved forward depth embedding.
max joint error (rad): 1.2932434082031252

I have to lift and strech out it's legs to make it trying to stand successfully.

I saw from other issues that I may need to use (L2+B then L2+A+START). However, I failed to use this. It seems that nothing changes when I press L2+A+START together and L2+A then START starts the Unitree's controller.

I have tried when the go1 is L2+A lockedd. Both low and high stand locking states lead to T_gru outputs as follows:

...
[ WARN] [1702138382.317291680]: Cmd lost, position control as motor damping. DO NOT use this as a normal operation.
[ERROR] [1702138382.327995616]: udp_recv error: -1, message translate might lost once.
[ERROR] [1702138382.437948096]: udp_recv error: -1, message translate might lost once.
[ERROR] [1702138382.547973920]: udp_recv error: -1, message translate might lost once.
[ERROR] [1702138382.657931168]: udp_recv error: -1, message translate might lost once.
[ERROR] [1702138382.767965280]: udp_recv error: -1, message translate might lost once.
[ERROR] [1702138382.877974912]: udp_recv error: -1, message translate might lost once.
...

I am not sure what's the problem. It seems like that the policy doesn't provide enough torque for the go1's stand-up evoking.

Thanks for any advices!

Hi, thanks for the amazing work.

I noticed that when deploying in the real world using Intel Realsense you start with a resolution of 240 X 424 and finally downsample it to 48 x 64, but in Isaac Gym you seem to start with an initial camera resolution of 240/4 x 424/4 and finally resize it to 48 x 64.

1. Why can't we directly take depth images of resolution 48 x 64 in Isaac Gym ? or even start with 240 x 424 as we do in the real world and then resize them to 48 x 64? (the 240/4 x 424/4 seems very counter-intuitive)
2. Is there any specific reason why you start with a 240 x 424 in the Realsense camera given it supports other resolutions?

TIA

I'm new to train a locomotion task. And the training process is complex. I have followed #3 to train the policy. But I still have some confusion on the training results. For example, I have tried to train the walk policy several times but it turned out to work badly as I used play.py to visualize the trained policy. And the walk policy training part is not mentioned in this paper so I'm wondering the training of walk is just a warmup or the result should be good. And should I just follow the steps in #3 ? Are there any ways to determine whether the training is useful so that I can relax the constraints or some other parameters to make the training process successful?

Thank you for open sourcing the code, much appreciated.
I am trying to deploy this on the Go1 and understand the camera height map code.
it seems to me that the height map part is not implemented on the version that is here. as when I am looking at the a1_real.py file the function _get_heights is a placeholder:

def _get_heights(self):
        """ TODO: get estimated terrain heights around the robot base """
        # currently return a zero tensor with valid size
        return torch.zeros(self.num_envs, 187, device= self.model_device, requires_grad= False)

also after reading some more, the camera sends the ros topic and thic code recieved it using this function _get_forward_depth_obs
but it is never called.

Am I missing something? could you share the complete code for generating the height map on the actual robot?
thanks.

May I ask when you will update the A1 deployment code

Hi,

Thank you very much for open-sourcing such a wonderful project. I have a simple question regarding the realsense camera you are using on Go1 robots. From the videos on your webpage, I don't think you mount cameras on A1 robots, so why mounting on Go1 robots? Is there something wrong with the onboard RGBD camera, or you are trying to mimic the A1 camera, or you just find out different viewing angle can help a lot for parkour?

Thanks a lot!

Hi, thank you for your great job about parkour. I encountered some problems about my GPU memory.It said that an illegal memory access was encountered.

Hello, when I use command line python legged_gym/legged_gym/scripts/train.py --task a1_down --headless to run a1_down or a1_jump, etc, there is a problem as follows:

Traceback (most recent call last):
  File "legged_gym/legged_gym/scripts/train.py", line 49, in <module>
    train(args)
  File "legged_gym/legged_gym/scripts/train.py", line 44, in train
    ppo_runner, train_cfg = task_registry.make_alg_runner(env=env, name=args.task, args=args, env_cfg=env_cfg)
  File "/root/parkour/legged_gym/legged_gym/utils/task_registry.py", line 171, in make_alg_runner
    resume_path = get_load_path(log_root, load_run=train_cfg.runner.load_run, checkpoint=train_cfg.runner.checkpoint)
  File "/root/parkour/legged_gym/legged_gym/utils/helpers.py", line 132, in get_load_path
    models = [file for file in os.listdir(load_run) if 'model' in file]
FileNotFoundError: [Errno 2] No such file or directory: '/root/parkour/legged_gym/logs/field_a1/{Your trained walk model directory}'

And I find the configure in class A1JumpCfgPPO.runner:

class runner( A1FieldCfgPPO.runner ):
        policy_class_name = "ActorCriticRecurrent"
        experiment_name = "field_a1"
        resume = True
        load_run = "{Your traind walking model directory}"
        load_run = "{Your virtually trained jump model directory}"

But I did not have a trained jump model and I can`t.

And when I use command line python legged_gym/legged_gym/scripts/train.py --task a1_field --headless, it dose works, but in every episode, the Mean Reward is 0. I can not find the reason.

Hi,

First of all, thank you for open-sourcing the awesome work.

I want to reproduce the policy but am unsure about the correct steps. This is my assumed procedure (I highlight things not clear):

1. Train specialized skills

• Walk (hard):
  python legged_gym/scripts/train.py --headless --task a1_field
• Climb (soft):
  virtual_terrain = False
  comment the class sensor
  update climb["depth"] (I'm not sure what you meant, the correct thing may be climb["height"]?)
  python legged_gym/scripts/train.py --headless --task a1_climb --load_run **{the dir of walk?}**
python legged_gym/scripts/train.py --headless --task a1_climb --load_run Oct15_14-21-57_WalkingBase_pEnergySubsteps2e-5_aScale0.5
• Climb (hard):
  virtual_terrain = True
  Uncomment the class sensor
  update climb["depth"] (I'm not sure what you meant, the correct thing may be climb["height"]?)
  python legged_gym/scripts/train.py --headless --task a1_climb --load_run **{the dir of previous soft climb?}**
python legged_gym/scripts/train.py --headless --task a1_climb --load_run Oct15_14-27-56_Skillclimb_pDofLimit-1e-01_climbHeight0.2-0.6_trackBlockLen1.6
• Leap (soft):
  virtual_terrain = False
  comment the class sensor
python legged_gym/scripts/train.py --headless --task a1_leap --load_run **{the dir of walk?}**
  python legged_gym/scripts/train.py --headless --task a1_leap --load_run Oct15_14-21-57_WalkingBase_pEnergySubsteps2e-5_aScale0.5
• Leap (hard):
  virtual_terrain = True
  Uncomment the class sensor
python legged_gym/scripts/train.py --headless --task a1_leap --load_run **{the dir of previous soft leap?}**
• Same steps for crawl and tilt

2. Distill the parkour policy

• Update the fields with my hard trained skill directories

sub_policy_paths = [ # must in the order of obstacle ID, Replace the folder name with your own training logdir
    os.path.join(logs_root, "field_a1/{your walking policy}"),
    os.path.join(logs_root, "field_a1/{your tilting policy}"),
    os.path.join(logs_root, "field_a1/{your crawling policy}"),
    os.path.join(logs_root, "field_a1/{your climbing policy}"),
    os.path.join(logs_root, "field_a1/{your leaping policy}"),
]
climb_down_policy_path = os.path.join(logs_root, "field_a1/{your climbing down policy}")

What is climb_down_policy_path?

• Run data collection (before or after training?)
  python legged_gym/scripts/collect.py --headless --task a1_distill (do I need load_run and what will it be)?
• Update the A1FieldDistillCfgPPO.runner.pretrain_dataset.data_dir field with a list of dataset directories. (Not sure how to do this?)
  Comment out the A1FieldDistillCfgPPO.runner.pretrain_dataset.scan_dir field.
• Run python legged_gym/scripts/train.py --headless --task a1_distill to start distillation (before or after collecting?)
• Comment out A1FieldDistillCfgPPO.runner.pretrain_dataset.data_dir field and uncomment A1FieldDistillCfgPPO.runner.pretrain_dataset.scan_dir field. (in a1_field_distill_config.py)
• Update the A1FieldDistillCfgPPO.runner.load_run field with your last distillation log.
• Run python legged_gym/scripts/train.py --headless --task a1_distill to start dagger. The terminal will prompt you to launch a collector process.
• To run the collector process, change RunnerCls to DaggerSaver (as in lines 20-21). Run python legged_gym/scripts/collect.py --headless --task a1_distill --load_run {the training run prompt by the training process}.
• Press Enter in the training process when you see some data is collected (prompted by the collector process). The training process will load the collected data and start training.

3. Visualization
   python legged_gym/scripts/play.py --task {task} --load_run **{dir of the log of corresponding task?}**

What are the differences between distill_{robot}_dagger and distill_{robot}?
Can I run distillation in one GPU?

Thanks

Hi, in the deploy file you show how to deploy code in go1,but how can i train the policy with go1 in vitural environment?

According to the operation procedure, the system reported an error, I tried many ways, but could not solve, I hope to get your help, thank you very much！

Similar to issue 7, the rewards of a1_field is always 0 and play.py repeatedly reset. Even if I add only_positive_reward = False in A1FieldCfg.rewards. But with the addition of this code, play.py will not restart again, and the robot dog will always stand in place, rather than walk forward. I'm wondering if I did something wrong?

These are the details:

1. IF I ADD only_positive_reward = False in a1_field_config.py like:

class rewards( A1RoughCfg.rewards ):

        class scales:

            tracking_ang_vel = 0.05

            world_vel_l2norm = -1.

            legs_energy_substeps = -2e-5

            legs_energy = -0.

            alive = 2.

            # penalty for hardware safety

            exceed_dof_pos_limits = -1e-1

            exceed_torque_limits_i = -2e-1

        soft_dof_pos_limit = 0.01

        only_positive_reward = True      # here I add the code 

after 10000 iterations, the dog can stand up with close to 100% probability like this:

2. IF I DON'T ADD the code, the play.py will repeatedly reset and have similar output in issue 7. I tried to comment the check part in the code like follow, but it doesn't work.

        # if dones.any():
        #     agent_model.reset(dones)
        #     print("env dones,{} because has timeout".format("" if env.time_out_buf[dones].any() else " not"))
        #     print(infos)

hi,
I am trying to train the policy with go1, but I do not know how to check whether the position of the front camera is in correct position. In other words, after setting a position in the config file, how can I know the position of the front camera in the Isaac gym environment?