opendilab / ace Goto Github PK

Maybe it's not appropriate to aks such question here. So if there is any bother, I'll apologize to you and won't put such issue any more.

I have read your paper, it's really elaborate. I think the way to embed the SE-state which contains actions, interactions and states is most suitable algorithm for my cooperative environment just like SMAC. But as a beginner in reinforcement learning, I feel it's so difficult for me to completely understand the whole project including hundred of files. I have read it for 2 days with just some frustration left. 😭

Could you give me a little tuition about which files should be emphatically read? And is the project not that suitable for directly using in other environment, or maybe self-implement is more suitable?

I ran your code and couldn't find where the win rate in SMAC are recorded.

I have a question about the results, how many samples were taken as the median? Also, how many moving averages are taken?
If you could let me know, that would be great.

/home/tj/.conda/envs/ace/lib/python3.8/site-packages/namedlist.py:252: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
if isinstance(field_names, _collections.Mapping):
Traceback (most recent call last):
File "exp/smac/corridor/config.py", line 4, in
from ding.entry import serial_pipeline
File "/home/tj/ACE/ding/entry/init.py", line 2, in
from .serial_entry import serial_pipeline
File "/home/tj/ACE/ding/entry/serial_entry.py", line 8, in
from ding.envs import get_vec_env_setting, create_env_manager
File "/home/tj/ACE/ding/envs/init.py", line 1, in
from .env import *
File "/home/tj/ACE/ding/envs/env/init.py", line 1, in
from .base_env import BaseEnv, get_vec_env_setting, BaseEnvTimestep, BaseEnvInfo, get_env_cls
File "/home/tj/ACE/ding/envs/env/base_env.py", line 14, in
class BaseEnv(ABC, gym.Env):
TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

This issue is a collection of various interesting agent demonstrations trained by ACE, it will be updated continually.

SMAC

We carefully analyze the behavior of transferred agents. As shown in the above video, ACE perfectly transfers the station position, careful positioning, alternating fire, and focusing fire of 5m vs 6m to 8m vs 9m and then achieves 40% win rate in 8m vs 9m. However, the optimal choice of focusing fire in 8m vs 9m is different from that in 5m vs 6m, which is largely responsible for ACE not achieving a 100% win rate on the 8m vs 9m map. We describe the transferring experiments in detail below

• ACE perfectly transfers the station position of 5m vs 6m to 8m vs 9m, making the agents stand up and down in a line.

• ACE perfectly transfers the operation of careful positioning and alternating fire of 5m vs 6m to 8m vs 9m. Each ally has very little health but kills all enemies.

• The optimal choice in 5m vs 6m is all allies focusing fire on one enemy. In comparison, the optimal choice in 8m vs 9m is all allies focusing fire on two enemies, since 5 allies can kill an enemy instantly, and having 8 agents attacking one agent at the same time is wasteful.

GRF

• academy_3_vs_1_with_keeper

• academy_counterattack_hard

"D:\Program Files\Anaconda3\envs\ACE\Scripts\python.exe" F:/Documents/ACE/exp/grf/academy_counterattack_hard/config.py WARNING:root:If you want to use numba to speed up segment tree, please install numba first D:\Program Files\Anaconda3\envs\ACE\lib\site-packages\torch\_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at ..\aten\src\ATen\native\BinaryOps.cpp:467.) return torch.floor_divide(self, other) INFO:learner_logger:[RANK0]: DI-engine DRL Policy GRFACE( (_action_encoder): Sequential( (0): Linear(in_features=128, out_features=128, bias=True) (1): ReLU(inplace=True) (2): Linear(in_features=128, out_features=2432, bias=True) (3): ReLU(inplace=True) ) (_state_encoder): Sequential( (0): Linear(in_features=21, out_features=128, bias=True) (1): ReLU(inplace=True) (2): Linear(in_features=128, out_features=128, bias=True) (3): ReLU(inplace=True) ) (_relation_encoder): Sequential( (0): Linear(in_features=132, out_features=128, bias=True) (1): ReLU(inplace=True) (2): Linear(in_features=128, out_features=256, bias=True) (3): ReLU(inplace=True) ) (_relation_aggregator): RelationAggregator( (_state_encoder): Sequential( (0): Linear(in_features=384, out_features=128, bias=True) (1): ReLU(inplace=True) ) ) (_decision_encoder): DecisionEncoder( (_decision_encoder): Sequential( (0): Linear(in_features=128, out_features=128, bias=True) (1): ReLU(inplace=True) (2): Linear(in_features=128, out_features=256, bias=True) (3): ReLU(inplace=True) ) (_logit_encoder): Sequential( (0): Linear(in_features=256, out_features=128, bias=True) (1): ReLU(inplace=True) (2): Linear(in_features=128, out_features=1, bias=True) ) ) (_local_predictor): Sequential( (0): Linear(in_features=128, out_features=64, bias=True) (1): ReLU(inplace=True) (2): Linear(in_features=64, out_features=7, bias=True) ) (_global_predictor): Sequential( (0): Linear(in_features=256, out_features=128, bias=True) (1): ReLU(inplace=True) (2): Linear(in_features=128, out_features=14, bias=True) ) ) WARNING:root:If you want to use numba to speed up segment tree, please install numba first WARNING:root:If you want to use numba to speed up segment tree, please install numba first WARNING:root:If you want to use numba to speed up segment tree, please install numba first D:\Program Files\Anaconda3\envs\ACE\lib\site-packages\torch\_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at ..\aten\src\ATen\native\BinaryOps.cpp:467.) return torch.floor_divide(self, other) D:\Program Files\Anaconda3\envs\ACE\lib\site-packages\torch\_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at ..\aten\src\ATen\native\BinaryOps.cpp:467.) return torch.floor_divide(self, other) D:\Program Files\Anaconda3\envs\ACE\lib\site-packages\torch\_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at ..\aten\src\ATen\native\BinaryOps.cpp:467.) return torch.floor_divide(self, other) WARNING:root:If you want to use numba to speed up segment tree, please install numba first WARNING:root:If you want to use numba to speed up segment tree, please install numba first WARNING:root:If you want to use numba to speed up segment tree, please install numba first WARNING:root:If you want to use numba to speed up segment tree, please install numba first D:\Program Files\Anaconda3\envs\ACE\lib\site-packages\torch\_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at ..\aten\src\ATen\native\BinaryOps.cpp:467.) return torch.floor_divide(self, other) WARNING:root:If you want to use numba to speed up segment tree, please install numba first D:\Program Files\Anaconda3\envs\ACE\lib\site-packages\torch\_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at ..\aten\src\ATen\native\BinaryOps.cpp:467.) return torch.floor_divide(self, other) D:\Program Files\Anaconda3\envs\ACE\lib\site-packages\torch\_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at ..\aten\src\ATen\native\BinaryOps.cpp:467.) return torch.floor_divide(self, other) D:\Program Files\Anaconda3\envs\ACE\lib\site-packages\torch\_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at ..\aten\src\ATen\native\BinaryOps.cpp:467.) return torch.floor_divide(self, other) D:\Program Files\Anaconda3\envs\ACE\lib\site-packages\torch\_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at ..\aten\src\ATen\native\BinaryOps.cpp:467.) return torch.floor_divide(self, other) WARNING:root:Timeout wrapper is not implemented in windows platform, so ignore it default WARNING:root:Timeout wrapper is not implemented in windows platform, so ignore it default WARNING:root:Timeout wrapper is not implemented in windows platform, so ignore it default WARNING:root:Timeout wrapper is not implemented in windows platform, so ignore it default WARNING:root:Timeout wrapper is not implemented in windows platform, so ignore it default WARNING:root:Timeout wrapper is not implemented in windows platform, so ignore it default WARNING:root:Timeout wrapper is not implemented in windows platform, so ignore it default WARNING:root:Timeout wrapper is not implemented in windows platform, so ignore it default F:\Documents\ACE\dizoo\gfootball\envs\gfootball_academy_env_ace.py:147: FutureWarning: In the future np.boolwill be defined as the corresponding NumPy scalar. self.action_mask = np.ones((self.left_player_num, self.action_len), dtype=np.bool) # all actions are valid F:\Documents\ACE\dizoo\gfootball\envs\gfootball_academy_env_ace.py:147: FutureWarning: In the futurenp.boolwill be defined as the corresponding NumPy scalar. self.action_mask = np.ones((self.left_player_num, self.action_len), dtype=np.bool) # all actions are valid F:\Documents\ACE\dizoo\gfootball\envs\gfootball_academy_env_ace.py:147: FutureWarning: In the futurenp.boolwill be defined as the corresponding NumPy scalar. self.action_mask = np.ones((self.left_player_num, self.action_len), dtype=np.bool) # all actions are valid F:\Documents\ACE\dizoo\gfootball\envs\gfootball_academy_env_ace.py:147: FutureWarning: In the futurenp.boolwill be defined as the corresponding NumPy scalar. self.action_mask = np.ones((self.left_player_num, self.action_len), dtype=np.bool) # all actions are valid F:\Documents\ACE\dizoo\gfootball\envs\gfootball_academy_env_ace.py:147: FutureWarning: In the futurenp.boolwill be defined as the corresponding NumPy scalar. self.action_mask = np.ones((self.left_player_num, self.action_len), dtype=np.bool) # all actions are valid F:\Documents\ACE\dizoo\gfootball\envs\gfootball_academy_env_ace.py:147: FutureWarning: In the futurenp.boolwill be defined as the corresponding NumPy scalar. self.action_mask = np.ones((self.left_player_num, self.action_len), dtype=np.bool) # all actions are valid F:\Documents\ACE\dizoo\gfootball\envs\gfootball_academy_env_ace.py:147: FutureWarning: In the futurenp.boolwill be defined as the corresponding NumPy scalar. self.action_mask = np.ones((self.left_player_num, self.action_len), dtype=np.bool) # all actions are valid F:\Documents\ACE\dizoo\gfootball\envs\gfootball_academy_env_ace.py:147: FutureWarning: In the futurenp.bool` will be defined as the corresponding NumPy scalar.
self.action_mask = np.ones((self.left_player_num, self.action_len), dtype=np.bool) # all actions are valid
ERROR:root:Function <function AsyncSubprocessEnvManager._reset..reset_fn at 0x000001FE15871AF0> has exceeded max retries(5)
ERROR:root:VEC_ENV_MANAGER: env 4 reset error
ERROR:root:Function <function AsyncSubprocessEnvManager._reset..reset_fn at 0x000001FE158719D0> has exceeded max retries(5)
ERROR:root:VEC_ENV_MANAGER: env 3 reset error
ERROR:root:Function <function AsyncSubprocessEnvManager._reset..reset_fn at 0x000001FE15871C10> has exceeded max retries(5)
ERROR:root:VEC_ENV_MANAGER: env 5 reset error
ERROR:root:
Env Process Reset Exception:
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 391, in _reset
reset_fn()
File "F:\Documents\ACE\ding\envs\env_manager\base_env_manager.py", line 56, in wrapper
raise runtime_error
File "F:\Documents\ACE\ding\envs\env_manager\base_env_manager.py", line 44, in wrapper
ret = func(*args, **kwargs)
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 383, in reset_fn
self._check_data({env_id: obs}, close=False)
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 630, in _check_data
raise exceptions[0]
RuntimeError('Function <function AsyncSubprocessEnvManager._reset..reset_fn at 0x000001FE15871C10> has exceeded max retries(5), and the latest exception is: AttributeError('\nEnv Process Exception:\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 601, in worker_fn_robust\n ret = reset_fn(*args, **kwargs)\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 586, in reset_fn\n raise e\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 579, in reset_fn\n ret = env.reset(*args, **kwargs)\n File "F:\\Documents\\ACE\\dizoo\\gfootball\\envs\\gfootball_academy_env_ace.py", line 119, in reset\n self._env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gym\\core.py", line 251, in reset\n return self.env.reset(**kwargs)\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gym\\core.py", line 278, in reset\n observation = self.env.reset(**kwargs)\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\wrappers.py", line 93, in reset\n return self.env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\football_env.py", line 186, in reset\n self._env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\football_env_core.py", line 117, in reset\n self._reset(self._env.game_config.render, inc=inc)\nAttributeError("\'NoneType\' object has no attribute \'game_config\'")')')
ERROR:root:Function <function AsyncSubprocessEnvManager._reset..reset_fn at 0x000001FE15871670> has exceeded max retries(5)
ERROR:root:VEC_ENV_MANAGER: env 0 reset error
ERROR:root:
Env Process Reset Exception:
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 391, in _reset
reset_fn()
File "F:\Documents\ACE\ding\envs\env_manager\base_env_manager.py", line 56, in wrapper
raise runtime_error
File "F:\Documents\ACE\ding\envs\env_manager\base_env_manager.py", line 44, in wrapper
ret = func(*args, **kwargs)
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 383, in reset_fn
self._check_data({env_id: obs}, close=False)
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 630, in _check_data
raise exceptions[0]
RuntimeError('Function <function AsyncSubprocessEnvManager._reset..reset_fn at 0x000001FE15871670> has exceeded max retries(5), and the latest exception is: AttributeError('\nEnv Process Exception:\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 601, in worker_fn_robust\n ret = reset_fn(*args, **kwargs)\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 586, in reset_fn\n raise e\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 579, in reset_fn\n ret = env.reset(*args, **kwargs)\n File "F:\\Documents\\ACE\\dizoo\\gfootball\\envs\\gfootball_academy_env_ace.py", line 119, in reset\n self._env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gym\\core.py", line 251, in reset\n return self.env.reset(**kwargs)\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gym\\core.py", line 278, in reset\n observation = self.env.reset(**kwargs)\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\wrappers.py", line 93, in reset\n return self.env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\football_env.py", line 186, in reset\n self._env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\football_env_core.py", line 117, in reset\n self._reset(self._env.game_config.render, inc=inc)\nAttributeError("\'NoneType\' object has no attribute \'game_config\'")')')
ERROR:root:Function <function AsyncSubprocessEnvManager._reset..reset_fn at 0x000001FE15871D30> has exceeded max retries(5)
ERROR:root:VEC_ENV_MANAGER: env 6 reset error
ERROR:root:
Env Process Reset Exception:
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 391, in _reset
reset_fn()
File "F:\Documents\ACE\ding\envs\env_manager\base_env_manager.py", line 56, in wrapper
raise runtime_error
File "F:\Documents\ACE\ding\envs\env_manager\base_env_manager.py", line 44, in wrapper
ret = func(*args, **kwargs)
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 383, in reset_fn
self._check_data({env_id: obs}, close=False)
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 630, in _check_data
raise exceptions[0]
RuntimeError('Function <function AsyncSubprocessEnvManager._reset..reset_fn at 0x000001FE15871D30> has exceeded max retries(5), and the latest exception is: AttributeError('\nEnv Process Exception:\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 601, in worker_fn_robust\n ret = reset_fn(*args, **kwargs)\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 586, in reset_fn\n raise e\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 579, in reset_fn\n ret = env.reset(*args, **kwargs)\n File "F:\\Documents\\ACE\\dizoo\\gfootball\\envs\\gfootball_academy_env_ace.py", line 119, in reset\n self._env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gym\\core.py", line 251, in reset\n return self.env.reset(**kwargs)\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gym\\core.py", line 278, in reset\n observation = self.env.reset(**kwargs)\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\wrappers.py", line 93, in reset\n return self.env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\football_env.py", line 186, in reset\n self._env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\football_env_core.py", line 117, in reset\n self._reset(self._env.game_config.render, inc=inc)\nAttributeError("\'NoneType\' object has no attribute \'game_config\'")')')
ERROR:root:Function <function AsyncSubprocessEnvManager._reset..reset_fn at 0x000001FE15871E50> has exceeded max retries(5)
ERROR:root:Function <function AsyncSubprocessEnvManager._reset..reset_fn at 0x000001FE158718B0> has exceeded max retries(5)
ERROR:root:
Env Process Reset Exception:
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 391, in _reset
reset_fn()
File "F:\Documents\ACE\ding\envs\env_manager\base_env_manager.py", line 56, in wrapper
raise runtime_error
File "F:\Documents\ACE\ding\envs\env_manager\base_env_manager.py", line 44, in wrapper
ret = func(*args, **kwargs)
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 383, in reset_fn
self._check_data({env_id: obs}, close=False)
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 630, in _check_data
raise exceptions[0]
RuntimeError('Function <function AsyncSubprocessEnvManager._reset..reset_fn at 0x000001FE158719D0> has exceeded max retries(5), and the latest exception is: AttributeError('\nEnv Process Exception:\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 601, in worker_fn_robust\n ret = reset_fn(*args, **kwargs)\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 586, in reset_fn\n raise e\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 579, in reset_fn\n ret = env.reset(*args, **kwargs)\n File "F:\\Documents\\ACE\\dizoo\\gfootball\\envs\\gfootball_academy_env_ace.py", line 119, in reset\n self._env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gym\\core.py", line 251, in reset\n return self.env.reset(**kwargs)\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gym\\core.py", line 278, in reset\n observation = self.env.reset(**kwargs)\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\wrappers.py", line 93, in reset\n return self.env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\football_env.py", line 186, in reset\n self._env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\football_env_core.py", line 117, in reset\n self._reset(self._env.game_config.render, inc=inc)\nAttributeError("\'NoneType\' object has no attribute \'game_config\'")')')
ERROR:root:
Env Process Reset Exception:
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 391, in _reset
reset_fn()
File "F:\Documents\ACE\ding\envs\env_manager\base_env_manager.py", line 56, in wrapper
raise runtime_error
File "F:\Documents\ACE\ding\envs\env_manager\base_env_manager.py", line 44, in wrapper
ret = func(*args, **kwargs)
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 383, in reset_fn
self._check_data({env_id: obs}, close=False)
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 630, in _check_data
raise exceptions[0]
RuntimeError('Function <function AsyncSubprocessEnvManager._reset..reset_fn at 0x000001FE15871AF0> has exceeded max retries(5), and the latest exception is: AttributeError('\nEnv Process Exception:\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 601, in worker_fn_robust\n ret = reset_fn(*args, **kwargs)\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 586, in reset_fn\n raise e\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 579, in reset_fn\n ret = env.reset(*args, **kwargs)\n File "F:\\Documents\\ACE\\dizoo\\gfootball\\envs\\gfootball_academy_env_ace.py", line 119, in reset\n self._env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gym\\core.py", line 251, in reset\n return self.env.reset(**kwargs)\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gym\\core.py", line 278, in reset\n observation = self.env.reset(**kwargs)\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\wrappers.py", line 93, in reset\n return self.env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\football_env.py", line 186, in reset\n self._env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\football_env_core.py", line 117, in reset\n self._reset(self._env.game_config.render, inc=inc)\nAttributeError("\'NoneType\' object has no attribute \'game_config\'")')')
ERROR:root:Function <function AsyncSubprocessEnvManager._reset..reset_fn at 0x000001FE15871790> has exceeded max retries(5)
ERROR:root:VEC_ENV_MANAGER: env 7 reset error
ERROR:root:VEC_ENV_MANAGER: env 2 reset error
ERROR:root:VEC_ENV_MANAGER: env 1 reset error
ERROR:root:
Env Process Reset Exception:
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 391, in _reset
reset_fn()
File "F:\Documents\ACE\ding\envs\env_manager\base_env_manager.py", line 56, in wrapper
raise runtime_error
File "F:\Documents\ACE\ding\envs\env_manager\base_env_manager.py", line 44, in wrapper
ret = func(*args, **kwargs)
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 383, in reset_fn
self._check_data({env_id: obs}, close=False)
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 630, in _check_data
raise exceptions[0]
RuntimeError('Function <function AsyncSubprocessEnvManager._reset..reset_fn at 0x000001FE158718B0> has exceeded max retries(5), and the latest exception is: AttributeError('\nEnv Process Exception:\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 601, in worker_fn_robust\n ret = reset_fn(*args, **kwargs)\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 586, in reset_fn\n raise e\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 579, in reset_fn\n ret = env.reset(*args, **kwargs)\n File "F:\\Documents\\ACE\\dizoo\\gfootball\\envs\\gfootball_academy_env_ace.py", line 119, in reset\n self._env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gym\\core.py", line 251, in reset\n return self.env.reset(**kwargs)\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gym\\core.py", line 278, in reset\n observation = self.env.reset(**kwargs)\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\wrappers.py", line 93, in reset\n return self.env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\football_env.py", line 186, in reset\n self._env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\football_env_core.py", line 117, in reset\n self._reset(self._env.game_config.render, inc=inc)\nAttributeError("\'NoneType\' object has no attribute \'game_config\'")')')
ERROR:root:
Env Process Reset Exception:
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 391, in _reset
reset_fn()
File "F:\Documents\ACE\ding\envs\env_manager\base_env_manager.py", line 56, in wrapper
raise runtime_error
File "F:\Documents\ACE\ding\envs\env_manager\base_env_manager.py", line 44, in wrapper
ret = func(*args, **kwargs)
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 383, in reset_fn
self._check_data({env_id: obs}, close=False)
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 630, in _check_data
raise exceptions[0]
RuntimeError('Function <function AsyncSubprocessEnvManager._reset..reset_fn at 0x000001FE15871790> has exceeded max retries(5), and the latest exception is: AttributeError('\nEnv Process Exception:\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 601, in worker_fn_robust\n ret = reset_fn(*args, **kwargs)\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 586, in reset_fn\n raise e\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 579, in reset_fn\n ret = env.reset(*args, **kwargs)\n File "F:\\Documents\\ACE\\dizoo\\gfootball\\envs\\gfootball_academy_env_ace.py", line 119, in reset\n self._env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gym\\core.py", line 251, in reset\n return self.env.reset(**kwargs)\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gym\\core.py", line 278, in reset\n observation = self.env.reset(**kwargs)\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\wrappers.py", line 93, in reset\n return self.env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\football_env.py", line 186, in reset\n self._env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\football_env_core.py", line 117, in reset\n self._reset(self._env.game_config.render, inc=inc)\nAttributeError("\'NoneType\' object has no attribute \'game_config\'")')')
Traceback (most recent call last):
ERROR:root:
Env Process Reset Exception:
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 391, in _reset
reset_fn()
File "F:\Documents\ACE\ding\envs\env_manager\base_env_manager.py", line 56, in wrapper
raise runtime_error
File "F:\Documents\ACE\ding\envs\env_manager\base_env_manager.py", line 44, in wrapper
ret = func(*args, **kwargs)
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 383, in reset_fn
self._check_data({env_id: obs}, close=False)
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 630, in _check_data
raise exceptions[0]
RuntimeError('Function <function AsyncSubprocessEnvManager._reset..reset_fn at 0x000001FE15871E50> has exceeded max retries(5), and the latest exception is: AttributeError('\nEnv Process Exception:\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 601, in worker_fn_robust\n ret = reset_fn(*args, **kwargs)\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 586, in reset_fn\n raise e\n File "F:\\Documents\\ACE\\ding\\envs\\env_manager\\subprocess_env_manager.py", line 579, in reset_fn\n ret = env.reset(*args, **kwargs)\n File "F:\\Documents\\ACE\\dizoo\\gfootball\\envs\\gfootball_academy_env_ace.py", line 119, in reset\n self._env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gym\\core.py", line 251, in reset\n return self.env.reset(**kwargs)\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gym\\core.py", line 278, in reset\n observation = self.env.reset(**kwargs)\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\wrappers.py", line 93, in reset\n return self.env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\football_env.py", line 186, in reset\n self._env.reset()\n File "D:\\Program Files\\Anaconda3\\envs\\ACE\\lib\\site-packages\\gfootball\\env\\football_env_core.py", line 117, in reset\n self._reset(self._env.game_config.render, inc=inc)\nAttributeError("\'NoneType\' object has no attribute \'game_config\'")')')
File "F:\Documents\ACE\ding\utils\system_helper.py", line 57, in run
self.ret = self._target(*self._args, **self._kwargs)
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 399, in _reset
raise e
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 391, in _reset
reset_fn()
File "F:\Documents\ACE\ding\envs\env_manager\base_env_manager.py", line 56, in wrapper
raise runtime_error
File "F:\Documents\ACE\ding\envs\env_manager\base_env_manager.py", line 44, in wrapper
ret = func(*args, **kwargs)
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 383, in reset_fn
self._check_data({env_id: obs}, close=False)
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 630, in _check_data
raise exceptions[0]
RuntimeError: Function <function AsyncSubprocessEnvManager._reset..reset_fn at 0x000001FE15871C10> has exceeded max retries(5), and the latest exception is: AttributeError('\nEnv Process Exception:\n File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 601, in worker_fn_robust\n ret = reset_fn(*args, **kwargs)\n File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 586, in reset_fn\n raise e\n File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 579, in reset_fn\n ret = env.reset(*args, **kwargs)\n File "F:\Documents\ACE\dizoo\gfootball\envs\gfootball_academy_env_ace.py", line 119, in reset\n self._env.reset()\n File "D:\Program Files\Anaconda3\envs\ACE\lib\site-packages\gym\core.py", line 251, in reset\n return self.env.reset(**kwargs)\n File "D:\Program Files\Anaconda3\envs\ACE\lib\site-packages\gym\core.py", line 278, in reset\n observation = self.env.reset(**kwargs)\n File "D:\Program Files\Anaconda3\envs\ACE\lib\site-packages\gfootball\env\wrappers.py", line 93, in reset\n return self.env.reset()\n File "D:\Program Files\Anaconda3\envs\ACE\lib\site-packages\gfootball\env\football_env.py", line 186, in reset\n self._env.reset()\n File "D:\Program Files\Anaconda3\envs\ACE\lib\site-packages\gfootball\env\football_env_core.py", line 117, in reset\n self._reset(self._env.game_config.render, inc=inc)\nAttributeError("'NoneType' object has no attribute 'game_config'")')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "F:\Documents\ACE\exp\grf\academy_counterattack_hard\config.py", line 102, in
train(args)
File "F:\Documents\ACE\exp\grf\academy_counterattack_hard\config.py", line 93, in train
serial_pipeline(config, seed=args.seed)
File "F:\Documents\ACE\ding\entry\serial_entry.py", line 61, in serial_pipeline
collector = create_serial_collector(
File "F:\Documents\ACE\ding\worker\collector\base_serial_collector.py", line 87, in create_serial_collector
return SERIAL_COLLECTOR_REGISTRY.build(cfg.type, cfg=cfg, **kwargs)
File "F:\Documents\ACE\ding\utils\registry.py", line 90, in build
raise e
File "F:\Documents\ACE\ding\utils\registry.py", line 76, in build
return build_fn(*obj_args, **obj_kwargs)
File "F:\Documents\ACE\ding\worker\collector\episode_serial_collector.py", line 62, in init
self.reset(policy, env)
File "F:\Documents\ACE\ding\worker\collector\episode_serial_collector.py", line 120, in reset
self.reset_env(_env)
File "F:\Documents\ACE\ding\worker\collector\episode_serial_collector.py", line 77, in reset_env
self._env.launch()
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 302, in launch
self.reset(reset_param)
File "F:\Documents\ACE\ding\envs\env_manager\subprocess_env_manager.py", line 357, in reset
t.join()
File "F:\Documents\ACE\ding\utils\system_helper.py", line 64, in join
raise RuntimeError('Exception in thread({})'.format(id(self))) from self.exc
RuntimeError: Exception in thread(2190794540800)

进程已结束,退出代码1
`