anujmahajanoxf / maven Goto Github PK
View Code? Open in Web Editor NEWSubmission for MAVEN: Multi-Agent Variational Exploration
Submission for MAVEN: Multi-Agent Variational Exploration
I pip installed the dependencies fromrequirements.txt
of PYMARL on python3.6 and I don't think there is a problem with dependencies, but I keep receiving the following error when I run the experiment python src/main.py --config=noisemix_episode --env-config=sc2 with env_args.map_name=3s5z
Traceback (most recent call last):
File "/usr/lib64/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib64/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/home/mop216/MAVEN/maven_code/src/runners/parallel_runner.py", line 311, in env_worker
env = env_fn.x()
File "/home/mop216/MAVEN/maven_code/src/envs/__init__.py", line 7, in env_fn
return env(**kwargs)
File "/home/mop216/MAVEN/maven_code/src/envs/starcraft2/starcraft2.py", line 142, in __init__
self._launch()
File "/home/mop216/MAVEN/maven_code/src/envs/starcraft2/starcraft2.py", line 206, in _launch
self._sc2_proc = self._run_config.start(version=self.game_version, window_size=self.window_size)
File "/home/mop216/MAVENN/lib64/python3.6/site-packages/pysc2/run_configs/platforms.py", line 205, in start
want_rgb=want_rgb, extra_args=extra_args, **kwargs)
File "/home/mop216/MAVENN/lib64/python3.6/site-packages/pysc2/run_configs/platforms.py", line 88, in start
self, exec_path=exec_path, version=self.version, **kwargs)
TypeError: type object got multiple values for keyword argument 'version'
Please note that --config=noise_qmix_parallel --env-config=nmatrix
configuration runs properly and uses nmatrix
environment. I think there goes something wrong with the starcraft2.py
which I cannot figure it out.
Hello,
Can you share MAVEN's .ymal file for the n_step matrix in the paper?
Some parameters are missing in the plot_keys/keys_matrix_games and I'm not sure my current setting is the same as yours.
Thank you very much!
Hi, which config and command should I use to train the results of Corridor?
Hi,
I just setup the environment and everything worked fine! Thanks for the great work~
But I wonder if you could provide us with the baseline model so we could use it without retraining. Because the training process on those super hard maps takes a really long time.
Thanks again and I am glad for your reply~
Hello,
Thanks for your codes. I'm running the codes with noisemix_smac.yaml in algs and sc2.yaml in envs.
However, it is hard to reproduce the MAVEN's results in the corridor and 6h_vs_8z.
Could you please provide the running configurations of MAVEN in your paper.
Thanks!
I am trying to reproduce your results and I have an issue running the code (not running on a docker). The backpropagation fails with the following error:
[ERROR 13:59:02] pymarl Failed after 0:00:23!
Traceback (most recent calls WITHOUT Sacred internals):
File "src/main.py", line 34, in my_main
run(_run, _config, _log)
File "/home/USER/.tmp/MAVEN/maven_code/src/run.py", line 48, in run
run_sequential(args=args, logger=logger)
File "/home/USER/.tmp/MAVEN/maven_code/src/run.py", line 181, in run_sequential
learner.train(episode_sample, runner.t_env, episode)
File "/home/USER/.tmp/MAVEN/maven_code/src/learners/noise_q_learner.py", line 168, in train
loss.backward()
File "/home/USER/.general_env/lib/python3.8/site-packages/torch/tensor.py", line 195, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/home/USER/.general_env/lib/python3.8/site-packages/torch/autograd/__init__.py", line 97, in backward
Variable._execution_engine.run_backward(
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [32, 56, 3, 9]], which is output 0 of SliceBackward, is at version 2; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
During handling of the above exception, another exception occurred:
Traceback (most recent calls WITHOUT Sacred internals):
File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__
self.gen.throw(type, value, traceback)
File "/usr/lib/python3.8/subprocess.py", line 1079, in wait
return self._wait(timeout=timeout)
File "/usr/lib/python3.8/subprocess.py", line 1796, in _wait
raise TimeoutExpired(self.args, timeout)
subprocess.TimeoutExpired: Command '['tee', '-a', '/tmp/tmp8fr28bt_']' timed out after 1 seconds
I have spent some time searching for the error and came up with the following:
This is what I changed to have an up and running code, in the train function of src/learners/noise_q_learner.py:
# Max over target Q-Values
if self.args.double_q:
mac_out_detach = mac_out.clone().detach()
mac_out_detach[avail_actions == 0] = -9999999
cur_max_actions = mac_out_detach[:, 1:].max(dim=3, keepdim=True)[1]
target_max_qvals = torch.gather(
target_mac_out, 3, cur_max_actions
).squeeze(3)
# Get actions that maximise live Q (for double q-learning)
#mac_out[avail_actions == 0] = -9999999
#cur_max_actions = mac_out[:, 1:].max(dim=3, keepdim=True)[1]
#target_max_qvals = th.gather(target_mac_out, 3, cur_max_actions).squeeze(3)
else:
target_max_qvals = target_mac_out.max(dim=3)[0]
# Discriminator
mac_out_detach = mac_out.clone().detach()
mac_out_detach[avail_actions == 0] = -9999999
q_softmax_actions = torch.nn.functional.softmax(
mac_out_detach[:, :-1], dim=3
)
#mac_out[avail_actions == 0] = -9999999
#q_softmax_actions = th.nn.functional.softmax(mac_out[:, :-1], dim=3)
Can you tell me if these changes are ok and if not how the gradient propagation should be fixed? I assume the proper way to fix the discriminator backprop problem would be to remove the part of the target that is in line with unavailable actions. I will keep looking for that in the code but I would really like to have your input.
Hi
Thanks for the very inspiring paper and code. Following your suggestion, I tried to run your code in the provided docker image. However, when I run
bash run.sh $GPU python3 src/main.py --config=noisemix_episode --env-config=sc2 with env_args.map_name=3s5z
,
I get error:
Launching container named 'iliu3_python3_1204202011_0_SlUA' on GPU '0' standard_init_linux.go:207: exec user process caused "exec format error"
.
I am able to run the code of pymarl (https://github.com/oxwhirl/pymarl) repo with the docker image they provided.
Thanks.
Hi, I tried to run the code on 6h_vs_8z
and failed. It shows something wrong in the starcraft2.py
. And I also found 6h_vs_8z
is not a registered game in your code. Could you please help me to fix it?
PySC2==3.0
Traceback (most recent call last):
File "/home/me/anaconda3/envs/sc_ray/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/home/me/anaconda3/envs/sc_ray/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "/home/me/maven/maven_code/src/runners/parallel_runner.py", line 333, in env_worker
env.reset()
File "/home/me/maven/maven_code/src/envs/starcraft2/starcraft2.py", line 270, in reset
return self.get_obs(), self.get_state()
File "/home/me/maven/maven_code/src/envs/starcraft2/starcraft2.py", line 878, in get_state
ally_state[al_id, 1] = al_unit.weapon_cooldown / max_cd # cooldown
TypeError: unsupported operand type(s) for /: 'float' and 'NoneType'
Hi, could you please help me to solve the problem? It's happened when I test code 'python3 src/main.py --config=noisemix_episode --env-config=sc2 with env_args.map_name=3s5z'
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [32, 132, 8, 14]], which is output 0 of SliceBackward, is at version 2; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
May I know the code of environment "3step" and "4step" ? I only find map file here but I do not know the reward and environment configuration of both map in neither paper or code.
Hi, I saw the following error when the code runs:
"Process Process-1:
Traceback (most recent call last):
File "/home/jovyan/miniconda3/envs/pytorch/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/jovyan/miniconda3/envs/pytorch/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/home/jovyan/MAVEN/maven_code/src/runners/parallel_runner.py", line 311, in env_worker
env = env_fn.x()
File "/home/jovyan/MAVEN/maven_code/src/envs/init.py", line 7, in env_fn
return env(**kwargs)
File "/home/jovyan/MAVEN/maven_code/src/envs/starcraft2/starcraft2.py", line 142, in init
self._launch()
File "/home/jovyan/MAVEN/maven_code/src/envs/starcraft2/starcraft2.py", line 206, in _launch
self._sc2_proc = self._run_config.start(version=self.game_version, window_size=self.window_size)
File "/home/jovyan/miniconda3/envs/pytorch/lib/python3.6/site-packages/pysc2/run_configs/platforms.py", line 205, in start
want_rgb=want_rgb, extra_args=extra_args, **kwargs)
File "/home/jovyan/miniconda3/envs/pytorch/lib/python3.6/site-packages/pysc2/run_configs/platforms.py", line 88, in start
self, exec_path=exec_path, version=self.version, **kwargs)
TypeError: type object got multiple values for keyword argument 'version'"
Do you have any idea on this? Also can you share your "requirement.txt" so I can know which version of the tool I should use? Many thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.