hungtuchen / pytorch-dqn Goto Github PK

Hello I get this error, how to fix it?

I have installed gym successfully, but I have this error.
-Thank you-

How do I test the model? Also what is the difference betweein ram.py and train.py ?

Are there any plans to update recent SOTA algorithms?

How much time did the training take you on games such as pong and breakout?

ubuntu 16.04
python3.5
anaconda
installed gym refered https://github.com/openai/gym instruction.
but when i run your code

Traceback (most recent call last):
File "/home/op/pytorch-dqn-master/main.py", line 3, in
import openai_benchmark
ImportError: No module named 'openai_benchmark'

i found in the search engine,
someone runs dir(gym),the result contains 'benchmark_spec', 'benchmarks',
but none on my pc.

Thanks in adance.

It's trivial though.

I run the code,and found your dqn algorithm take such a long time to converge.Actually,I found few implementation of dqn can converge in github.They can converge in a afternoon.I use a piece of GTX1080Ti.It is appreciate that your implementation can converge.But your code take a day and a night to converge.I don't know why.

I'm running pytorch 0.2,

and the code dqn_learn.py fail to work..

the error as follow

Traceback (most recent call last):
  File "ram.py", line 57, in <module>
    main(env)
  File "ram.py", line 46, in main
    target_update_freq=TARGER_UPDATE_FREQ,
  File "/auto/master05/ssarcandy/ttt/dqn_learn.py", line 213, in dqn_learing
    current_Q_values.backward(d_error.data.unsqueeze(1))
  File "/home/master/05/ssarcandy/.local/lib/python2.7/site-packages/torch/autograd/variable.py", line 156, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, retain_variables)
  File "/home/master/05/ssarcandy/.local/lib/python2.7/site-packages/torch/autograd/__init__.py", line 98, in backward
    variables, grad_variables, retain_graph)
  File "/home/master/05/ssarcandy/.local/lib/python2.7/site-packages/torch/autograd/function.py", line 91, in apply
    return self._forward_cls.backward(self, *args)
  File "/home/master/05/ssarcandy/.local/lib/python2.7/site-packages/torch/autograd/_functions/tensor.py", line 566, in backward
    return grad_input.scatter_add_(ctx.dim, index, grad_output), None, None
  File "/home/master/05/ssarcandy/.local/lib/python2.7/site-packages/torch/autograd/variable.py", line 696, in scatter_add_
    return ScatterAdd.apply(self, dim, index, source, True)
  File "/home/master/05/ssarcandy/.local/lib/python2.7/site-packages/torch/autograd/_functions/tensor.py", line 605, in forward
    return input.scatter_add_(ctx.dim, index, source)
RuntimeError: invalid argument 3: Index tensor must have same dimensions as input tensor at /pytorch/torch/lib/THC/generic/THCTensorScatterGather.cu:198

Hi, thanks for sharing your wonderful code.
But I have met some errors when running it.

1. Inside the line 197~205 from dqn_learn.py, the size of target_Q_values and that of current_Q_values does not matched well. I have changed to next_max_q = next_max_q.unsqueeze(-1) for correcting sizes. Also I have changed to rew_batch[0] from line 203.

2. (IMO) After stacking records in replay buffer, queue action does not work properly. I have changed the line 158 to action = select_epilson_greedy_action(Q, recent_observations, t), however different action value has queued.

I am still working these but having troubles. Could you help make them right?