unity-technologies / paper-ml-agents

A repository to reproduce the experiments of the MA-POCA paper

papers-ml-agents

This repository is meant to offer a way to reproduce the experiments of some of the papers Unity published using ML-Agents.

MA-POCA

Multi-Agent POsthumous Credit Assignment (MA-POCA) is a multi-agent learning algorithm that can gracefully deal with spawning and despawning agents without the use of absorbing states. See here for more information.

paper-ml-agents's Issues

Independent actors configuration

Hi, I want to know if there's any setting in POCA where each agent in a team gets a separate policy with a separate neural network. Can I change any parameters in the configuration yaml file

HTTP Error Launching poca environment

Hi There,

I'm seeing the Unity Splash Screen and have followed the install and run instructions as best as I can tell. I'm running into an error where it seems to be trying to connect to a URL which no longer exists. Am I missing something or should I be running differently?

Version information:
ml-agents: 0.1,
ml-agents-envs: 0.27.0,
Communicator API: 1.5.0,
PyTorch: 2.0.0+cpu
[WARNING] Attempt 1 / 5: Failed to download and extract binary.
[WARNING] Attempt 2 / 5: Failed to download and extract binary.
[WARNING] Attempt 3 / 5: Failed to download and extract binary.
[WARNING] Attempt 4 / 5: Failed to download and extract binary.
[ERROR] UnityEnvironment worker 0: environment raised an unexpected exception.
Traceback (most recent call last):
File "d:\development\paper-ml-agents-main\paper-ml-agents-main\ma-poca\mapoca\mapoca\trainers\subprocess_env_manager.py", line 160, in worker
env = env_factory(worker_id, side_channels)
File "d:\development\paper-ml-agents-main\paper-ml-agents-main\ma-poca\mapoca\mapoca\trainers\learn.py", line 174, in create_unity_environment
return mapoca_registry[env_name].make(
File "D:\Downloads\anaconda3\envs\mlagents20\lib\site-packages\mlagents_envs\registry\remote_registry_entry.py", line 61, in make
path = get_local_binary_path(self.identifier, url)
File "D:\Downloads\anaconda3\envs\mlagents20\lib\site-packages\mlagents_envs\registry\binary_utils.py", line 40, in get_local_binary_path
download_and_extract_zip(url, name)
File "D:\Downloads\anaconda3\envs\mlagents20\lib\site-packages\mlagents_envs\registry\binary_utils.py", line 131, in download_and_extract_zip
request = urllib.request.urlopen(url, timeout=30)
File "D:\Downloads\anaconda3\envs\mlagents20\lib\urllib\request.py", line 214, in urlopen
return opener.open(url, data, timeout)
File "D:\Downloads\anaconda3\envs\mlagents20\lib\urllib\request.py", line 523, in open
response = meth(req, response)
File "D:\Downloads\anaconda3\envs\mlagents20\lib\urllib\request.py", line 632, in http_response
response = self.parent.error(
File "D:\Downloads\anaconda3\envs\mlagents20\lib\urllib\request.py", line 561, in error
return self._call_chain(*args)
File "D:\Downloads\anaconda3\envs\mlagents20\lib\urllib\request.py", line 494, in _call_chain
result = func(*args)
File "D:\Downloads\anaconda3\envs\mlagents20\lib\urllib\request.py", line 641, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found https://storage.googleapis.com/mlagents-test-environments/ma-poca/win/Startup.zip
[INFO] Learning was interrupted. Please wait while the graph is generated.
Traceback (most recent call last):
File "D:\Downloads\anaconda3\envs\mlagents20\lib\multiprocessing\queues.py", line 245, in _feed
obj = _ForkingPickler.dumps(obj)
File "D:\Downloads\anaconda3\envs\mlagents20\lib\multiprocessing\reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
TypeError: cannot pickle '_io.BufferedReader' object
Process Process-1:
Traceback (most recent call last):
File "d:\development\paper-ml-agents-main\paper-ml-agents-main\ma-poca\mapoca\mapoca\trainers\subprocess_env_manager.py", line 160, in worker
env = env_factory(worker_id, side_channels)
File "d:\development\paper-ml-agents-main\paper-ml-agents-main\ma-poca\mapoca\mapoca\trainers\learn.py", line 174, in create_unity_environment
return mapoca_registry[env_name].make(
File "D:\Downloads\anaconda3\envs\mlagents20\lib\site-packages\mlagents_envs\registry\remote_registry_entry.py", line 61, in make
path = get_local_binary_path(self.identifier, url)
File "D:\Downloads\anaconda3\envs\mlagents20\lib\site-packages\mlagents_envs\registry\binary_utils.py", line 40, in get_local_binary_path
download_and_extract_zip(url, name)
File "D:\Downloads\anaconda3\envs\mlagents20\lib\site-packages\mlagents_envs\registry\binary_utils.py", line 131, in download_and_extract_zip
request = urllib.request.urlopen(url, timeout=30)
File "D:\Downloads\anaconda3\envs\mlagents20\lib\urllib\request.py", line 214, in urlopen
return opener.open(url, data, timeout)
File "D:\Downloads\anaconda3\envs\mlagents20\lib\urllib\request.py", line 523, in open
response = meth(req, response)
File "D:\Downloads\anaconda3\envs\mlagents20\lib\urllib\request.py", line 632, in http_response
response = self.parent.error(
File "D:\Downloads\anaconda3\envs\mlagents20\lib\urllib\request.py", line 561, in error
return self._call_chain(*args)
File "D:\Downloads\anaconda3\envs\mlagents20\lib\urllib\request.py", line 494, in _call_chain
result = func(*args)
File "D:\Downloads\anaconda3\envs\mlagents20\lib\urllib\request.py", line 641, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found https://storage.googleapis.com/mlagents-test-environments/ma-poca/win/Startup.zip

Thanks,
Ram