TypeError: stack(): argument 'tensors' (position 1) must be tuple of Tensors, not collections.deque about rainbow

[Kaixhin]

For starters, which version of Python are you using (Python 3 should be fine) and what version of PyTorch (v0.3.0 or v0.3.1 should be fine)?

[forhonourlx]

Hi Kaixhin,
I am using windows 10 Python 3.5, PyTorch version may be 0.4.0 which is complied by https://github.com/peterjc123/pytorch-scripts

Thanks for your help.

[Kaixhin]

Not really sure what the problem is. Can you replace line 53 in env.py with return torch.stack(list(self.state_buffer), 0) to see if that works? If it doesn't can you print out self.state_buffer and paste the result here?

[forhonourlx]

With "list(self.state_buffer)", it does work.
self.state_buffer: deque([
0 0 0 ... 0 0 0
0 0 0 ... 0 0 0
0 0 0 ... 0 0 0
... ?? ...
0 0 0 ... 0 0 0
0 0 0 ... 0 0 0
0 0 0 ... 0 0 0
[torch.cuda.FloatTensor of size (84,84) (GPU 0)]
,
0 0 0 ... 0 0 0
0 0 0 ... 0 0 0
0 0 0 ... 0 0 0
... ?? ...
0 0 0 ... 0 0 0
0 0 0 ... 0 0 0
0 0 0 ... 0 0 0
[torch.cuda.FloatTensor of size (84,84) (GPU 0)]
,
0 0 0 ... 0 0 0
0 0 0 ... 0 0 0
0 0 0 ... 0 0 0
... ?? ...
0 0 0 ... 0 0 0
0 0 0 ... 0 0 0
0 0 0 ... 0 0 0
[torch.cuda.FloatTensor of size (84,84) (GPU 0)]
,
0.0000 0.0000 0.0000 ... 0.0000 0.0000 0.0000
0.0000 0.0000 0.0000 ... 0.0000 0.0000 0.0000
0.0000 0.0000 0.0000 ... 0.0000 0.0000 0.0000
... ?? ...
0.3098 0.3098 0.3098 ... 0.3098 0.3098 0.3098
0.3098 0.3098 0.3098 ... 0.3098 0.3098 0.3098
0.3098 0.3098 0.3098 ... 0.3098 0.3098 0.3098
[torch.cuda.FloatTensor of size (84,84) (GPU 0)]
], maxlen=4)
(84,84) (GPU 0)]
], maxlen=4)

What's more, another exception occurs:

Traceback (most recent call last):
File "main.py", line 103, in
next_state, reward, done = env.step(action) # Step
File "C:\Users\simon\Desktop\DQN\RL-AlphaGO\Rainbow-master\env.py", line 63, in step
reward += self.ale.act(self.actions.get(action))
File "C:\Program Files\Python35\lib\site-packages\atari_py\ale_python_interface.py", line 159, in act
return ale_lib.act(self.obj, int(action))
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

[forhonourlx]

......
self.actions.get(action): 4
self.actions.get(action): 4
self.actions.get(action): 4
self.actions.get(action): 4
self.actions.get(action): 1
self.actions.get(action): 1
self.actions.get(action): 1
self.actions.get(action): 1
self.actions.get(action): None
Traceback (most recent call last):
File "main.py", line 103, in
next_state, reward, done = env.step(action) # Step
File "C:\Users\simon\Desktop\DQN\RL-AlphaGO\Rainbow-master\env.py", line 63, in step
reward += self.ale.act(self.actions.get(action))
File "C:\Program Files\Python35\lib\site-packages\atari_py\ale_python_interface.py", line 159, in act
return ale_lib.act(self.obj, int(action))
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

[Kaixhin]

That's a different problem, so could you open a separate issue for that (so I can track this specific problem separately)?

Before I close this issue I think there may be a bigger problem you have with PyTorch. Try running the following code, and if you get an error then you should post it as a PyTorch issue.

from collections import deque
import torch
x = deque([torch.ones(3), torch.ones(3)])
torch.stack(x)

[forhonourlx]

from collections import deque
import torch
x = deque([torch.ones(3), torch.ones(3)])
torch.stack(x)
Traceback (most recent call last):
File "", line 1, in
TypeError: stack(): argument 'tensors' (position 1) must be tuple of Tensors, not collections.deque

torch.stack(list(x))
1 1 1
1 1 1
[torch.FloatTensor of size (2,3)]

Thanks. I will post it as a PyTorch issue.