Global Navigation: Does the robot know the next waypoint by setting the reward function？ about gdae

[reiniscimurs]

Hi,

Why are you adding the reward function to this code? Are you planning to continue training in this setting with already pre-trained model?

Be aware that there are differences in how state is represented here in GDAE and in the base DRL training repository. Mainly, the robot state.

[Yangzuodong]

Hi,
Thanks for your reply！

Firstly, I don't want to continue training, I want to make the robot move entirely through TD3's actor network, without relying on move_base. But I don't know how the agent knows local_goal. After looking at the test_velodyne_td3.py file, I guessed that the robot knew the end point because it was set up in env to receive a reward for reaching the end point.

Secondly, I modified the state representation in GDAM to be consistent with that in TD3. This includes laser_state and robot_state

Lastly，Thank you very much for your patient reply！

[reiniscimurs]

Reward is not necessary for model deployment. It is only needed to train the model. The model knows the goal as it is part of the state that is given to the model.

There is no distinction between local and global goal. At each individual step the model recieves a single target to go to and it does not know if it is a global or local goal. The step function reads the current selected node and uses it as the current target that is then passed to the TD3 model in the state. The node selection is done entirely through the heuristics function.

[Yangzuodong]

Hi，
Thanks for your reply！So does the robot know the target by the Dist_to_goal parameter in the robot_state returned from step function？
I modified the code and now it looks like the video：

freecompress-GDAM_4_9-ezgif.com-video-cutter.mp4

However, the motion path of the agent is a little strange, it seems that it always wants to move clockwise.This was not the case with test_velodeny_td3.

Thank you very much for your work and your patient response！

[reiniscimurs]

The goal is given in polar coordinates by distance and angle. This is explained in the tutorial: https://medium.com/@reinis_86651/deep-reinforcement-learning-in-mobile-robot-navigation-tutorial-part3-training-13b2875c7b51

[Yangzuodong]

Thank you for solving my doubts, I wish you a happy life！