Ask for help to solve the problem of increasing memory. This part of the code has only been modified a little, and the memory will keep increasing during the training process. about deepbots-tutorials

[HongYegg]

更改了模型和trainstep中的state

[tsampazk]

Hello @HongYegg. Thank you for bringing this to our attention.

I can notice a minor increase in memory consumption of the python process on the task manager. It seems to come from the PPOAgent train step as you demostrate in your code using tracemalloc.

This increase is of an order of 1-2 mb after 500-1000 episodes of training. The agent solves the problem after 1000-1500 episodes usually. The python process starts at 94.4 mb.

To be honest it seems really minor and out of the scope of this tutorials repo. As far as i've used this implementation in the past it never seemed to cause any memory issues.

There are many discussions online 1, 2, 3 about increasing memory consuption during training when using PyTorch and can be caused by many different things as far as i can tell.