Comments (8)
Yes it's different. I remember you can set centered=True
in PyTorch to make them the same. But I'm not sure.
Do you mean frame skip? It exists in atari wrapper.
from deeprl.
Not only frame skip. I mean during the training process, you don't have to calculate gradient and then do bp every step.
By the way, you don't implement any clip for gradient?
from deeprl.
I mean during the training process, you don't have to calculate gradient and then do bp every step.
I don't know this trick, do you know where it comes from?
By the way, you don't implement any clip for gradient?
In my understanding gradient clip is only necessary in multi-process setting
from deeprl.
update_freq=4 defined here
https://github.com/kuz/DeepMind-Atari-Deep-Q-Learner/blob/1d52e98152fccd968d63ca67e33f5cbf3e83a878/run_gpu#L14
and used here
https://github.com/kuz/DeepMind-Atari-Deep-Q-Learner/blob/1d52e98152fccd968d63ca67e33f5cbf3e83a878/dqn/NeuralQLearner.lua#L341
clip_delta=1
https://github.com/kuz/DeepMind-Atari-Deep-Q-Learner/blob/1d52e98152fccd968d63ca67e33f5cbf3e83a878/run_gpu#L31
from deeprl.
Thanks, I haven't read the original lua code before.
It's interesting to know that tricks.
from deeprl.
Another issue................. In deepmind code, gradient clip is really "clip"
if self.clip_delta then
delta[delta:ge(self.clip_delta)] = self.clip_delta
delta[delta:le(-self.clip_delta)] = -self.clip_delta
end
But you use nn.utils.clip.grad_norm, which is more likely to do "normalization" .....
http://pytorch.org/docs/master/_modules/torch/nn/utils/clip_grad.html
from deeprl.
Yes .. I know that, just to conform with pytorch.
from deeprl.
Ok, thanks! So many details................
from deeprl.
Related Issues (20)
- Reference for the network design HOT 2
- Question about DeepRL-QUOTA-discrete
- Question about QUOTA-discrete HOT 4
- How to obtain the policy(logits) rather than only the actions from self.actor.step() HOT 2
- Passing info from Actor to Agent in Async implementation?
- Continuous Control Reward and State Normalization
- config parameters
- Does the DAC code apply to robotics environment? HOT 1
- Double constraints for updating actor?
- replay buffer: why next state start index is n_steps after state start index?
- SAC example?
- no module named baselines HOT 2
- How to get averaged curve of PPO online performance on Mujoco? HOT 2
- I want to get the experimental data used to plot, if you are willing to HOT 2
- using target network to calculate last state value HOT 4
- Add baselines to requirements.txt HOT 1
- How much rewards can we get using DQN to play atari games usually?
- dqn_pixel doesn't work under the multiprocess setting of "config.num_workers = 5" HOT 1
- Asynchronous DQN HOT 4
- Adding elements to the Transition namedtuple HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deeprl.