Runtime Error: Lost reference to actor about muzero-pytorch

[noanabeshima]

It works. I'm embarrassed, I didn't install the dependencies! Thanks for the swift answer.

[koulanurag]

Hi @noanabeshima ,

I just did a quick run of the code and it works fine. I guess you are facing an error due to missing dependencies and I can recommend following steps for investigation at your end:

1. install all dependencies in a fresh virtualenv environment. Please, make sure that you have correct versions of the required dependencies.[ReadMe]
2. Try to run the code with only 1 actor. [ref]
3. Try ray.init(local mode=True) [ref]. This will run the code in sync. mode and may help to find the issue.

Feel free to write back in case you still couldn't resolve it.

Log of Run on my Mac(no cuda):

(muzero-pytorch-new) (base) anuragkoul@Anurags-MacBook-Pro muzero-pytorch % python main.py --env CartPole-v1 --case classic_control --opr train --force
2020-05-09 22:52:15,687 INFO resource_spec.py:205 -- Starting Ray with 1.17 GiB memory available for workers and up to 0.61 GiB for objects. You can adjust these settings with ray.init(memory=, object_store_memory=).
2020-05-09 22:52:16,307 WARNING worker.py:348 -- WARNING: Falling back to serializing objects of type <class 'method'> by using pickle. This may be inefficient.
2020-05-09 22:52:16,398 WARNING worker.py:348 -- WARNING: Falling back to serializing objects of type <class 'range'> by using pickle. This may be inefficient.
(pid=15873) 2020-05-09 22:52:42,022 WARNING worker.py:348 -- WARNING: Falling back to serializing objects of type <class 'numpy.dtype'> by using pickle. This may be inefficient.
(pid=15873) 2020-05-09 22:52:42,032 WARNING worker.py:348 -- WARNING: Falling back to serializing objects of type <class 'numpy.random.mtrand.RandomState'> by using pickle. This may be inefficient.
[2020-05-09 22:52:42,393][train][INFO][train.py>_log] ==> #0 Loss: 38.378 [weighted Loss:38.378 Policy Loss: 4.107 Value Loss: 22.281 Reward Loss: 11.989 ] Replay Episodes Collected: 1 Buffer Size: 1 Lr: 0.050
[2020-05-09 22:52:42,394][train_test][INFO][train.py>_log] ==> #0 Test Score: 10.0
[2020-05-09 22:52:42,471][train][INFO][train.py>_log] ==> #1 Loss: 37.956 [weighted Loss:37.956 Policy Loss: 4.113 Value Loss: 22.029 Reward Loss: 11.814 ] Replay Episodes Collected: 1 Buffer Size: 1 Lr: 0.050
[2020-05-09 22:52:43,270][train][INFO][train.py>_log] ==> #2 Loss: 37.062 [weighted Loss:37.062 Policy Loss: 4.006 Value Loss: 21.574 Reward Loss: 11.481 ] Replay Episodes Collected: 1 Buffer Size: 1 Lr: 0.050
[2020-05-09 22:52:43,369][train][INFO][train.py>_log] ==> #3 Loss: 35.917 [weighted Loss:35.917 Policy Loss: 3.965 Value Loss: 20.937 Reward Loss: 11.015 ] Replay Episodes Collected: 1 Buffer Size: 1 Lr: 0.050
[2020-05-09 22:52:43,770][train][INFO][train.py>_log] ==> #4 Loss: 34.573 [weighted Loss:34.573 Policy Loss: 4.082 Value Loss: 20.044 Reward Loss: 10.447 ] Replay Episodes Collected: 1 Buffer Size: 1 Lr: 0.050
[2020-05-09 22:52:43,904][train][INFO][train.py>_log] ==> #5 Loss: 32.865 [weighted Loss:32.865 Policy Loss: 4.051 Value Loss: 19.039 Reward Loss: 9.776 ] Replay Episodes Collected: 1 Buffer Size: 1 Lr: 0.050
[2020-05-09 22:52:44,215][train][INFO][train.py>_log] ==> #6 Loss: 30.923 [weighted Loss:30.923 Policy Loss: 4.084 Value Loss: 17.796 Reward Loss: 9.042 ] Replay Episodes Collected: 1 Buffer Size: 1 Lr: 0.050
[2020-05-09 22:52:44,621][train][INFO][train.py>_log] ==> #7 Loss: 28.720 [weighted Loss:28.720 Policy Loss: 3.953 Value Loss: 16.571 Reward Loss: 8.196 ] Replay Episodes Collected: 1 Buffer Size: 1 Lr: 0.050