shangtongzhang / distributedes Goto Github PK

Looking through the code, it doesn't appear you you've used fitness shaping to help mitigate early local minimas in your NES algo. It's a fairly simple transform and Salimans et al. found it key to performance in their paper.

Hi,
I ran experiments on natural_es.py for the BipedalWalker environment and I see that the learning curve is not consistent at all. Most of the time, the value saturates around -100. I tried different network architecture and activation units. I hardly see 1/2 of 5 runs actually learning. Did anyone face the such situation? The learning curves for other environments look as expected.
It would be helpful if someone could share some points/hints to make BipedalWalker-v2 learn.
Any help is appreciated.
Thanks

I get this error when I run this code.Can you help me ?Thanks!
D:\Python\lib\site-packages\gym\envs\registration.py:14: PkgResourcesDeprecationWarning: Parameters to load are deprecated. Call .resolve and .require separately.
result = entry_point.load(False)
WARN: gym.spaces.Box autodetected dtype as <class 'numpy.float32'>. Please provide explicit dtype.
WARN: gym.spaces.Box autodetected dtype as <class 'numpy.float32'>. Please provide explicit dtype.
WARN: gym.spaces.Box autodetected dtype as <class 'numpy.float32'>. Please provide explicit dtype.
WARN: gym.spaces.Box autodetected dtype as <class 'numpy.float32'>. Please provide explicit dtype.
Traceback (most recent call last):
File "e:\VScode project\rl_ga\DistributedES\natural_es.py", line 154, in
all_tasks()
File "e:\VScode project\rl_ga\DistributedES\natural_es.py", line 150, in all_tasks
for p in ps: p.start()
File "D:\Python\lib\multiprocessing\process.py", line 105, in start
self._popen = self._Popen(self)
File "D:\Python\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "D:\Python\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "D:\Python\lib\multiprocessing\popen_spawn_win32.py", line 65, in init
reduction.dump(process_obj, to_child)
File "D:\Python\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'PendulumConfig.init..'
Traceback (most recent call last):
File "", line 1, in
File "D:\Python\lib\multiprocessing\spawn.py", line 99, in spawn_main
new_handle = reduction.steal_handle(parent_pid, pipe_handle)
File "D:\Python\lib\multiprocessing\reduction.py", line 87, in steal_handle
_winapi.DUPLICATE_SAME_ACCESS | _winapi.DUPLICATE_CLOSE_SOURCE)
PermissionError: [WinError 5] 鎷掔粷璁块棶銆?