nuria95 / o-raac Goto Github PK

Dear authors,

In you paper, there are customised reward functions for Half-Cheetah, Walker2D and Hopper. How can I find it in this repo?

Thank you!

Hi,

Thanks for providing this interesting package.

We are very interesting in your paper Risk-Averse Offline Reinforcement Learning and would like to replicate the results therein. However, I encounter two problem in trying to replicate the results.

1. In the two of the json files at O-RAAC/json_params/ORAAC/ (Params_cheetah_expert.json and Params_cheetah_medium.json) you have "RISK_DISTORTION": "wang", on other json files you have "RISK_DISTORTION": "cvar". However, in your paper you use CVaR_{0.1} as the risk measure. May I ask whether I should replace the two "RISK_DISTORTION": "wang" with "RISK_DISTORTION": "cvar" as in other four json files? Are there any other parameters in the json files that we need to change in order to replicate Table 2 in your paper ?

2. Does this package support training the model with GPU? I do see a line of code parser.add_argument('--device', action="store", type=str) in O-RAAC/utils/utilities.py/ but this argument seems used nowhere. I try to run you code in a GPU server using the command python3.8 oraac.py --device "cuda:0" --config_name 'Params_cheetah_medium.json' and the GPU seems NOT used. (I move the oraac.py file out from the experiments folder). Could you please let me know how did you train the models using GPUs.