TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType' about rainbow

[Kaixhin]

I see you're printing self.actions.get(action) and it's returning None. Can you also print action as well (just beforehand). Also print self.actions once (can do that at the end of the __init__ function in Env)? Which game are you using (default is Space Invaders)?

[forhonourlx]

Yes, the default is 'space_invaders'.

at init self.actions: {0: 0, 1: 1, 2: 3, 3: 4, 4: 11, 5: 12}
...
action: 5 ,self.actions.get(action): 12 ,done: False
action: 5 ,self.actions.get(action): 12 ,done: False
action: 5 ,self.actions.get(action): 12 ,done: False
action: 5 ,self.actions.get(action): 12 ,done: False
action: 1 ,self.actions.get(action): 1 ,done: False
action: 1 ,self.actions.get(action): 1 ,done: False
action: 1 ,self.actions.get(action): 1 ,done: False
action: 1 ,self.actions.get(action): 1 ,done: False
action: 3 ,self.actions.get(action): 4 ,done: False
action: 3 ,self.actions.get(action): 4 ,done: False
action: 3 ,self.actions.get(action): 4 ,done: False
action: 3 ,self.actions.get(action): 4 ,done: False
action: 1 ,self.actions.get(action): 1 ,done: False
action: 1 ,self.actions.get(action): 1 ,done: False
action: 1 ,self.actions.get(action): 1 ,done: False
action: 1 ,self.actions.get(action): 1 ,done: False
action: 3 ,self.actions.get(action): 4 ,done: False
action: 3 ,self.actions.get(action): 4 ,done: False
action: 3 ,self.actions.get(action): 4 ,done: False
action: 3 ,self.actions.get(action): 4 ,done: False
action: 1 ,self.actions.get(action): 1 ,done: False
action: 1 ,self.actions.get(action): 1 ,done: False
action: 1 ,self.actions.get(action): 1 ,done: False
action: 1 ,self.actions.get(action): 1 ,done: False
action:
2
[torch.cuda.LongTensor of size () (GPU 0)]
,self.actions.get(action): None ,done: False

[Kaixhin]

This seems like another change introduced by PyTorch 0.4. There's a lot of breaking changes so for now I would suggest downgrading to 0.3.1 if possible. Once 0.4 is released (I believe very soon) then I'll start working on porting the code.

[forhonourlx]

Hi Kaixhin, thank you all the way.
Sorry about my unfimilarity with PyTorch.
It's hard to find some keras implemenation of Rainbow DQN, and does it represent the heat of frameworks in some extent?
What's the advantages of PyTorch in your opinion?

[Kaixhin]

PyTorch is very Pythonic - you can write programs normally and introduce deep learning components. This means its much easier to deal with interacting with an environment, storing data in data structures etc. Becoming familiar with programming in Python in general, as well as going through the PyTorch tutorials, can help you become more familiar with it.