compatibility issue with latest gym (0.26.1) and stable -baseline3 (2.0.0)[ about citylearn

[kingsleynweye]

Hi @Ganesh-mali, there is a related issue to reference for this #63. You need to downgrade your stable-baselines3 installation, preferably to 1.7.0 or 1.8.0. CityLearn is still a gym environment whereas later stable-baselines3 releases support gymnasium environments.

[Ganesh-mali]

hey @kingsleynweye thank you for your reply, I tried downgrading to both 1.7.0 as well as 1.8.0. I get new error with that :

Building wheels for collected packages: gym
Building wheel for gym (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [477 lines of output]

Let me know , if you need more info.

[kingsleynweye]

@Ganesh-mali this error is a recently known gym bug for version 0.21. I have been able to bypass it in the past by downgrading pip and setuptools before installing CityLearn and stable-baselines3:

pip install pip==21.3.1
pip install setuptools==41.2.0
pip install CityLearn==2.0b4
pip install stable-baselines3==1.7.0

[kingsleynweye]

@Ganesh-mali hmmnn I have not seen this before. If I understand correctly, it is the stable-baselines3 installation throwing this new error? Also, looks like you are having issues with anaconda virtual environments. I am not very familiar with using anaconda as I typically use venv to manage virtual environments.

[Ganesh-mali]

hey @kingsleynweye , this time I created a virtual environment using Venv and repeated your steps, Again same error, the first three ran successfully and the installation of stable-baseline3 failed again.

Installing collected packages: zipp, gym, importlib-metadata, stable-baselines3
  Attempting uninstall: gym
    Found existing installation: gym 0.26.2
    Uninstalling gym-0.26.2:
      Successfully uninstalled gym-0.26.2
  DEPRECATION: gym is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
  Running setup.py install for gym ... error
  error: subprocess-exited-with-error

  × Running setup.py install for gym did not run successfully.
  │ exit code: 1
  ╰─> [573 lines of output]
      running install
      running build
      running build_py
      creating build

  This above message followed some 200 line output then it showed :

      , line 2449, in resolve
          module = __import__(self.module_name, fromlist=['__name__'], level=0)
        File "C:\COURSES_LEARNINGS\Rl_learning\city_learn\city_test\lib\site-packages\setuptools\command\bdist_wininst.py", line 1, in <module>
          import distutils.command.bdist_wininst as orig
      ModuleNotFoundError: No module named 'distutils.command.bdist_wininst'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  Rolling back uninstall of gym
  Moving to c:\courses_learnings\rl_learning\city_learn\city_test\lib\site-packages\gym-0.26.2.dist-info\
   from C:\COURSES_LEARNINGS\Rl_learning\city_learn\city_test\Lib\site-packages\~ym-0.26.2.dist-info
  Moving to c:\courses_learnings\rl_learning\city_learn\city_test\lib\site-packages\gym\
   from C:\COURSES_LEARNINGS\Rl_learning\city_learn\city_test\Lib\site-packages\~ym
error: legacy-install-failure

If you need more info please let me know. Or If you have any other suggestions, I will follow your guidelines I just want to run the code that is on Citylearn documentation.

[Ganesh-mali]

Hi @kingsleynweye

I have an update, I have installed all for packages as you said, After running the code I am getting this error now :

C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\torch\nn\modules\rnn.py:89: UserWarning: Failed to initialize NumPy: module compiled against API version 0x10 but this version of numpy is 0xe (Triggered internally at  ..\torch\csrc\utils\tensor_numpy.cpp:68.)
  w_ih = Parameter(torch.empty((gate_size, layer_input_size), **factory_kwargs))
Traceback (most recent call last):
  File "C:\COURSES_LEARNINGS\Rl_learning\city_learn\p1-city_learn.py", line 12, in <module>
    model.learn(total_timesteps=env.time_steps*20)
  File "C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\stable_baselines3\sac\sac.py", line 299, in learn
    return super().learn(
  File "C:\Users\\anaconda3\envs\new_city_test_env\lib\site-packages\stable_baselines3\common\off_policy_algorithm.py", line 323, in learn
    total_timesteps, callback = self._setup_learn(
  File "C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\stable_baselines3\common\off_policy_algorithm.py", line 305, in _setup_learn
    return super()._setup_learn(
  File "C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\stable_baselines3\common\base_class.py", line 408, in _setup_learn
    self._last_obs = self.env.reset()  # pytype: disable=annotation-type-mismatch
  File "C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\stable_baselines3\common\vec_env\dummy_vec_env.py", line 74, in reset
    obs = self.envs[env_idx].reset()
  File "C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\stable_baselines3\common\monitor.py", line 83, in reset
    return self.env.reset(**kwargs)
  File "C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\gym\core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\gym\core.py", line 379, in reset
    obs, info = self.env.reset(**kwargs)
  File "C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\gym\core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\gym\core.py", line 323, in reset
    return self.env.reset(**kwargs)
  File "C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\gym\core.py", line 379, in reset
    obs, info = self.env.reset(**kwargs)
ValueError: not enough values to unpack (expected 2, got 1)

[Ganesh-mali]

Hey @kingsleynweye any update on this ?

[H-Park]

This is indeed a bug. The library needs to depend on gymnasium, not gym. The OpenAI's gym package hasn't been maintained in years, and a independent developer took it upon themself to revive it and add features the RL community wanted, and in the process releases it on pypi as gymnasium to avoid issues.

[kingsleynweye]

@Ganesh-mali this part of the error:

obs, info = self.env.reset(**kwargs)
ValueError: not enough values to unpack (expected 2, got 1)

makes me believe the issue is caused by your installation of stable-baselines3 expecting a gymnasium environment where the reset function returns a tuple of two values. In line with what @H-Park mentioned, CityLearn is a gym environment and gym environments' step function returns a single value, obs.

To avoid this problem, you need to use a stable-baselines3 version compatible with gym (I think any stable-baselines3<=1.8.0).

[kingsleynweye]

This is indeed a bug. The library needs to depend on gymnasium, not gym. The OpenAI's gym package hasn't been maintained in years, and a independent developer took it upon themself to revive it and add features the RL community wanted, and in the process releases it on pypi as gymnasium to avoid issues.

@H-Park yeah, ideally, we would upgrade CityLearn to gymnasium and it is something on the list but not quite a priority yet :( . If this is a fix you will want to see sooner and have to know-how, please feel free to contribute as CityLearn is open source and I think migrating to gymnasium is a very welcomed community contribution through a pull request.

[kingsleynweye]

@Ganesh-mali this part of the error:

obs, info = self.env.reset(**kwargs)
ValueError: not enough values to unpack (expected 2, got 1)

makes me believe the issue is caused by your installation of stable-baselines3 expecting a gymnasium environment where the reset function returns a tuple of two values. In line with what @H-Park mentioned, CityLearn is a gym environment and gym environments' step function returns a single value, obs.

To avoid this problem, you need to use a stable-baselines3 version compatible with gym (I think any stable-baselines3<=1.8.0).

Forgot to mention that another way I have been able to solve this problem is by installing shimmy that provides a gymnasium binder.

[Ganesh-mali]

@kingsleynweye the dataset given in the example code "baeda_3dem" doesn't work, I even checked using the method Datset.get_names(). It printed all available datasets except baeda_3dem. Anyways, it is working with other datasets, I am able to run the code with SB3 (1.7.0), and gym (0.21.0). Thanks for all your suggestions.

[kingsleynweye]

@Ganesh-mali, if baeda_3dem is not getting printed then I suspect you are using a citylearn version that is <=1.8.0. It was only added after 1.8.0 It prints out for me for later versions.