Comments (5)
Hey, thanks for raising this! Will take care of shortly
from rlgraph.
Irrespective of the issue just a short piece of info on why the preprocessor code is even there:
The reason the preprocessors are done separately in Python is because we noticed an issue around image resizing in TensorFlow (https://hackernoon.com/how-tensorflows-tf-image-resize-stole-60-days-of-my-life-aba5eb093f35), so to do any benchmarks around images, we realised we would need to have the option to do the preprocessing with CV2 out-of-graph.
from rlgraph.
Cont.:
The heart of the issue is then that agent.reset() is meant to reset the preprocessor, but because of this separation as a consequence of this critical TF bug, it is not being called currently.
In any case, the name reset() is maybe misleading because users could reasonably expect that reset() fully resets the internal state (i.e. reinitalises all variables in particular).
We should hence maybe have a reset() method to fully reset agent state, and an episode_reset() which resets preprocessor state.
from rlgraph.
Since agent.reset() is not being called because of an external bug, I assume the worker(s) logic would remain the same until the TF bug is fixed? The reason I asked is that I put some logic into my modified agent and an end-of-episode agent.reset() call becomes critical in training.
Thanks for the reply and clarification!
from rlgraph.
Yes since preprocessing is being handled via the python/numpy preprocessor stack implementations by default now, the tf preprocessor does not need resetting (even thought it would not hurt). So you could add a reset call if your agent needed some specific extra resetting.
from rlgraph.
Related Issues (20)
- [ Agents ] Separate out root component definition of Agents.
- [Algorithm] SAC: Add discrete actions HOT 1
- Support for MARL HOT 3
- [Algorithms] Container actions for SAC HOT 1
- Using a custom gym HOT 4
- Agent stats logging HOT 7
- Checkpoints not created HOT 4
- [Algorithms] Support fully customised baseline networks HOT 1
- [API] Describe full kwargs in agents, not **kwargs. HOT 1
- [Tests] CI cleanup.
- [SAC]Agent returns the highest action value always HOT 1
- [Core] A Component should keep a registry for its optimizers. HOT 1
- [Core] Make it optional for Components to describe which input-args they need in order to create their variables.
- [Algorithms] Change return values of `action_from_preprocessed_state`.
- [Core] Move update controls and logic from Worker to Agent(s). HOT 2
- Add stateful-support to LSTMLayers.
- Inconsistence of the record key name in pytorch mode HOT 4
- Is apex pong broken? HOT 2
- Plug Pycolab envs into rlgraph
- problem
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rlgraph.