nikhilbarhate99 / td3-pytorch-bipedalwalker-v2

Twin Delayed DDPG (TD3) PyTorch solution for Roboschool and Box2d environment

TD3-BipedalWalker-v2-PyTorch

PyTorch implementation of Twin Delayed DDPG (TD3) tested on the following environments:

• Bipedal Walker v2

• Lunar Lander Continuous v2

• Walker2d v1 (Roboschool)

• HalfCheetah v1 (Roboschool)

Usage

• To test a preTrained network : run test.py

• To train a new network : run train.py

Dependencies

Trained and tested on:

Python 3.6
PyTorch 0.4.1
NumPy 1.15.3
gym 0.10.8
Roboschool 1.0.46
Pillow 5.3.0

Results

BipedalWalker-v2 (800 episodes)	LunarLanderContinuous-v2 (1500 episodes)

RoboschoolWalker2d-v1 (lr=0.002, 1400 episodes)	HalfCheetah-v1 (lr=0.002, 1400 episodes)

*The results are not consistent for BipedalWalker-v2 env

References

• Official TD3 paper and code

• OpenAI SpinningUp in Deep RL

td3-pytorch-bipedalwalker-v2's Issues

Lunar Lander hyperparameters

May I ask what are your hyperparameters for lunar lander?

Consistent results ?

Hey ! Thanks for open-sourcing your implementation, it's good to see RL examples in PyTorch.

I've been using your work for some tests and I wanted to know how consistent was your policy ? I mean, it sometimes trains successfully while in some runs completely fails.

What do you think ?

Thanks (:

Solved Bipedal Walker Environment

I noticed in your code you only check if the average of the last 10 episodes are above 300. But on the leaderboard page it requires that the last 100 episode average is above 300. Did you test it with such a large averaging window? The reason I ask is because from the figures in this #2 it appears that the reward signal is not that stable to get such a high average. I've been trying to solve the same environment with DDPG and the agent will master it but make a few mistakes in between making it hard to get the 100-episode average above 300.

Also did you ever experience the agent forgetting what it has learned after training the model longer?