zisikons / deep-rl Goto Github PK

I'm a PhD student at the University of New Mexico and I very much like your paper above. I tied to implement equation 5 in your paper for a more complex environment but most of the time the solution provided by the solver was infeasible even for the soft constraint. I wanted to know if you ran into that issue and how did you remedy it? My environment consist of 4 agents navigating a close area to reach a target location while avoiding inter agent collision and collision with 5 obstacles. I noticed that if I used just 3 agents and no obstacles I get better solution from the solver of course but no where close to what you showed in the paper. Albeit I used PPO instead of MADDPG. I'm wondering if you tried your implementation on more complex environment and what limitations did you notice? Are there any parameter tuning you can suggest. On major difference with my implementation is that I normalized the action space, observation and constraints. I'm wondering if you did the same?

By studying the paper it is not clear to me whether the agents learn to satisfy the constraints eventually. As I understand the safety layer optimization is always applied. Are the agents informed that they took an action that violated a constraint so that they can learn it somehow?