Comments (4)
I had the same problem and could solve it by removing the division by 255 (https://github.com/mlech26l/ncps/blob/master/examples/atari_torch.py#L122)
Seems like the observation already get scaled inside some function of one of the dependencies.
from ncps.
I have updated the Conv layers in the examples (by adding batch norms between the conv layers as suggested here). I get a reward > 40 after a couple of minutes of training (with pytorch behavior cloning)
The RL training probably benefits from these changes as well (+ RL needs to train for 24h or so to collect enough experience)
loss=0.3713: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 938/938 [01:43<00:00, 9.04it/s]
Epoch 1, val_loss=0.2822, val_acc=89.86%
Mean return 5.7 (n=10)
loss=0.2131: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 938/938 [01:42<00:00, 9.14it/s]
Epoch 2, val_loss=0.215, val_acc=92.41%
Mean return 20.1 (n=10)
loss=0.1761: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 938/938 [01:42<00:00, 9.18it/s]
Epoch 3, val_loss=0.1937, val_acc=93.19%
Mean return 71.8 (n=10)
loss=0.1563: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 938/938 [01:42<00:00, 9.17it/s]
Epoch 4, val_loss=0.1684, val_acc=94.15%
Mean return 60.2 (n=10)
loss=0.1426: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 938/938 [01:42<00:00, 9.17it/s]
Epoch 5, val_loss=0.154, val_acc=94.53%
Mean return 31.6 (n=10)
loss=0.1311: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 938/938 [01:42<00:00, 9.13it/s]
Epoch 6, val_loss=0.1506, val_acc=94.63%
Mean return 74.9 (n=10)
loss=0.1213: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 938/938 [01:42<00:00, 9.13it/s]
Epoch 7, val_loss=0.1624, val_acc=94.10%
Mean return 65.2 (n=10)
loss=0.1135: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 938/938 [01:42<00:00, 9.14it/s]
Epoch 8, val_loss=0.1443, val_acc=94.89%
Mean return 64.8 (n=10)
loss=0.1055: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 938/938 [01:42<00:00, 9.14it/s]
Epoch 9, val_loss=0.1376, val_acc=95.14%
Mean return 64.1 (n=10)
loss=0.09882: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████| 938/938 [01:42<00:00, 9.13it/s]
Epoch 10, val_loss=0.1359, val_acc=95.26%
Mean return 69.6 (n=10)
loss=0.09216: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████| 938/938 [01:42<00:00, 9.16it/s]
Epoch 11, val_loss=0.1377, val_acc=95.25%
Mean return 77.5 (n=10)
loss=0.08617: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████| 938/938 [01:42<00:00, 9.12it/s]
from ncps.
Many thanks for your reply!But my result remains the same using the new code with BN. I suppose there're some version issues in my Python environment. Could you kindly provide your conda environment info and the model's parameters (pytorch behavior cloning, several epochs will suffice). It would be really helpful for my reproduction work.
from ncps.
Indeed, I check the observation given by the env and find it already standardized to [0, 1), which do not conform with the description of Gym's documentation([0, 255]). However, this seems only to happen in Python 3.10. Removing observation's division by 255 can perfectly solve the problem (for Python version 3.10). Thanks for your solution!
from ncps.
Related Issues (20)
- Example of stacking LTC with convolutional layers on pytorch version HOT 2
- Inquiry on LTC Models for Financial Time-Series Prediction HOT 2
- Issues on recurrent connections in command layer for CfC
- getting issues while saving model HOT 2
- Input dimension HOT 1
- LtC and CfC implementation questions
- Pytorch model behaves differently after saving HOT 1
- How to define Output Dimension in NCP/LTC network?
- Example for image sequence classifier HOT 3
- Defining equal input and output shapes for LTC HOT 1
- Python gives anRuntime error when I try to send hidden state (hx) to the CfC model. HOT 2
- way to get 200G dataset
- What dependency versions? HOT 4
- module 'tensorflow.keras.layers' has no attribute 'AbstractRNNCell' HOT 3
- Error when using AutoNCP
- "Neural Circuit Policing to Enable Auditable Autonomy" Implementation Resource Request Please
- Clarification on Sensory and Motor Neurons in LTC Layers
- how to to increase the number of intern-neurons
- Save hidden state after training the model to initialize the initial hidden state of the model during inference HOT 1
- How to increase # of Layers with CfC or LTC Cell
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ncps.