Hyperparameter tuning using Custom env with RL-zoo and SB2 about rl-baselines-zoo

[araffin]

as a quick remark, please use SB3 and the associated RL zoo (cf readme), as SB2 is no longer actively developped.
And please fill the issue template properly.

[shivam-saurav]

I am trying to tune the hyperparameters but the error says :
Trial 2 failed because of the following error: AttributeError("'A2C' object has no attribute 'env'").

However, I have checked with check_env(),it works well with that.

check_env(env, warn=True)
(It works fine)
Output for this:
/usr/local/lib/python3.7/dist-packages/gym/logger.py:30: UserWarning:

WARN: Box bound precision lowered by casting to float32

Instantiate the agent using :
model = A2C('MlpPolicy', env, verbose=1)

And trying to tune the hyperparameter using :

hyperparams={}
data_frame = hyperparam_optimization('a2c', model, model.get_env(), n_trials=100,
n_timesteps=5000, hyperparams=hyperparams,
n_jobs=2, seed=0,
sampler_method='random', pruner_method='median',
verbose=1)

The error is :

[I 2021-05-14 06:39:38,453] Trial 1 finished with value: 800.7374267578125 and parameters: {'gamma': 0.995, 'n_steps': 32, 'lr_schedule': 'constant', 'lr': 0.00026730495810056736, 'ent_coef': 1.0750232421652547e-05, 'vf_coef': 0.3385992446532201}. Best is trial 1 with value: 800.7374267578125.
[W 2021-05-14 06:39:38,469] Trial 2 failed because of the following error: AttributeError("'A2C' object has no attribute 'env'")
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/optuna/_optimize.py", line 217, in run_trial
value_or_values = func(trial)
File "", line 104, in objective
eval_freq = max(eval_freq // model.get_env().num_envs, 1)
File "/usr/local/lib/python3.7/dist-packages/stable_baselines/common/base_class.py", line 104, in get_env
return self.env

AttributeError: 'A2C' object has no attribute 'env'

Just to clarify few things, I used the hyperparam_optimization function(from https://github.com/araffin/rl-baselines-zoo/blob/master/utils/hyperparams_opt.py) with few changes in the objective function.

Line 98 was changed to : model = model_fn #from model = model_fn(**kwargs)
Line 100 was changed to : eval_freq_ = eval_freq #from env_fn(n_envs=1, eval_env=True)

I understand that rl-baselines-zoo can be installed and required command can be run but that too requires the gym registry of the custom-env which was creating issue. I just want to clarify what's wrong in doing this and why the error is 'A2C' object has no attribute 'env' when A2C atually has.

[araffin]

Just to clarify few things, I used the hyperparam_optimization function(from https://github.com/araffin/rl-baselines-zoo/blob/master/utils/hyperparams_opt.py) with few changes in the objective function.

then please check that the script works without your modification and a normal env. If so, then you know where the issue come from ;)
(we don't do tech support as mentioned explicitly in the issue template and README)

but that too requires the gym registry of the custom-env which was creating issue.

why is that an issue?
https://github.com/DLR-RM/rl-baselines3-zoo#custom-environment

PS: please use markdown to format your code and traceback (cf issue template)