koulanurag / muzero-pytorch Goto Github PK

[2020-05-09 22:42:48,652][root][ERROR][main.py>] ==> Lost reference to actor
Traceback (most recent call last):
File "main.py", line 80, in
train(muzero_config, summary_writer)
File "/home/noa/cloned/muzero-pytorch/core/train.py", line 343, in train
for rank in range(0, config.num_actors)]
File "/home/noa/cloned/muzero-pytorch/core/train.py", line 343, in
for rank in range(0, config.num_actors)]
File "/home/noa/anaconda3/envs/torch/lib/python3.7/site-packages/ray/actor.py", line 103, in remote
return self._remote(args, kwargs)
File "/home/noa/anaconda3/envs/torch/lib/python3.7/site-packages/ray/actor.py", line 123, in _remote
return invocation(args, kwargs)
File "/home/noa/anaconda3/envs/torch/lib/python3.7/site-packages/ray/actor.py", line 112, in invocation
raise RuntimeError("Lost reference to actor")
RuntimeError: Lost reference to actor

Hi, I am just reading the original Muzero paper and find one detail that keeps me confused. Can I discuss it here?
In the paper, authors said:

We use a discrete support set of size 601 with one support for every integer between −300 and 300.

I wonder why they choose this support set? Does it have to do with the specific game environment? If I got another game environment other than Atari, (e.g, a game with a maximum reward of 10 at the end of the game). how should I choose the support set?

I noticed that in the requirements.txt file ray 1.13.0 is given. But the code is not compatible with this version, notable during training here:

ray.wait(workers, len(workers))

wait() only accepts one positional parameter and even if I use ray.wait(workers, num_returns=len(workers)), I get the following error:

wait() expected a list of ray.ObjectRef, got list containing <class 'ray.actor.ActorHandle'>

Still searching for a solution, but if somebody has a good idea, that would be awesome.

in main.py, there is if args.case == 'atari': from config.atari import muzero_config,
but there's no atari.py in config folder, just classic_control.py. is this file missing?