Comments (4)
Hi @MoMe36 , thank you for your positive feedback ! Regarding your questions:
- Can you explain what do you mean visual inspection here ? For current implementation, it checkpoints the model a few times with
.pth
file, and the file name is the number of iterations. If you mean generating video animation by execution a checkpoint file for one episode, this is done in baselines/plot.ipynb file, in the second cell, there ismake_video
function.
To make it more clear how the files are structured when one uses run_experiment
:
Suppose one defines experiment name as 'default' in the run_experiment
function, and there is no configuration sweeping, i.e. single configuration (it means no Grid/Random
object in the config), then a single unique job ID is number 0, and it has 3 random runs with different random seeds, say [123, 456, 789].
Then the file structure under logs looks like:
- logs
- default # experiment name
- 0 # job ID
- 123 # random seed
- 456
- 789
Under each seed leaf folder, all loggings, checkpoints are stored. To generate a video animation by executing one checkpoint file, one could load simply call make_video
function in the plot.ipynb
file with corresponding arguments, all the file loadings, action selection, generate mp4 files are done for you internally.
- For current status, the checkpointing mechanism is implemented, in the file
experiment.py
for each algorithm, you can seecheckpoint.num
in the config object to define how many checkpoint files to generate during entire training.- However, the resume training functionality is not supported yet for now. Although this should not be too painful to add, but because of the philosophy of this repo is to provide research-friendly code with minimialism and easy-to-modify. I am still hesitating what is the best way to implement it with minimal change plus good user-experience and low coding complexity. But for sure, this would be very nice functionality to have.
Hope it helps, and don't hesitate to discuss together if you have further questions.
from lagom.
Hi ! Thanks for the quick and detailed answer! I did figure out how to use the make_video
method which is exactly what I was looking for. However, I'm not able to find the checkpoints and logging.
I do have the folder structure you're describing, but when I check within a seed folder, I find only agent_1.pth
(along with obs_moment.pth
in the case of PPO). And the agent really doesn't perform as it should, given its episode returns.
I checked the config file within which I specified the log freq to be at 10, but I don't see any other checkpoint beside the one I mentionned, even after 200 iterations.
Do you have any idea what I'm doing wrong ? Thanks a lot (:
from lagom.
Alright, I get it ! Checkpoints are saved only at the end of training, it seems. Thanks anyway !
from lagom.
Hi @MoMe36 , for PPO, it's better to use logs/default
folder, because other logging folders are temporary (i.e. they might not have all checkpoints).
log_freq
: it only controls how frequently to dump loggings to the screen only.- The checkpointing is controlled by
checkpoint.num
, e.g. forcheckpoint.num=3
, it firstly checkpoints before the training, and second in the middle of the entire training, and third one at the end of the training. If you want more checkpoints, you can simply increase the integer in the config object.
Hope it helps !
from lagom.
Related Issues (20)
- VAE Experiment uses old run_experiments function HOT 4
- the repo is very heavy HOT 5
- discrete RL examples HOT 1
- make baselines part of lagom HOT 7
- make mujoco an optional dependence HOT 2
- Numerical bug in RunningMeanVar
- Remove lz4 + atari-related when merged upstream to gym
- Merge seed and device into config object
- Replace from_numpy/tensorify with as_tensor, remove tensorify
- Integrate Ray as an additional backend for parallel experiments
- Migrate obs/reward normalization from env.wrappers into Agent itself
- Support Python 3.8
- Refactor Hyperparameter classes
- Logging: convert loaded loggings entirely into pandas `DataFrame`
- Extract independent snapshot functions for metrics: network, RL, experiment, statistics etc.
- Agent constructor: replace environment object as spec object
- Agent.choose_action: replace the mode string argument with internal self.training
- Use more fair RL evaluation metrics: IQM, Optimality Gap, Median, Mean
- [setup.py] get_version: replace `importlib` with raw text scanning
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lagom.