Comments (1)
How to set a seed in a bsuite environment instance? In the notebook, the output of
sweep.SETTINGS
has a seed attribute which is not None:Loaded bsuite_id: bandit_noise/0. bsuite_id=bandit_noise/0, settings={'noise_scale': 0.1, 'seed': 0}, num_episodes=10000 Loaded bsuite_id: bandit_noise/1. bsuite_id=bandit_noise/1, settings={'noise_scale': 0.1, 'seed': 1}, num_episodes=10000 Loaded bsuite_id: bandit_noise/2. bsuite_id=bandit_noise/2, settings={'noise_scale': 0.1, 'seed': 2}, num_episodes=10000 Loaded bsuite_id: bandit_noise/3. bsuite_id=bandit_noise/3, settings={'noise_scale': 0.1, 'seed': 3}, num_episodes=10000
but when I printed it again on my own computer, seed was
None
(if I do that in the notebook, seed was None but there's an extramapping_seed
which was not None).I tried two methods to seed the environment: (1)
sweep.SETTINGS[bsuite_id]['seed']=0
; (2) doingenv.seed()
after wrapping it with OpenAI env, but neither worked (multiple experiments, same seed, different results). A minimal example to demonstrate these two seeding methods are not working:import random import torch as t import numpy as np import bsuite from bsuite import sweep from bsuite.utils import gym_wrapper def set_seed(seed, deterministic=True): random.seed(seed) np.random.seed(seed) t.manual_seed(seed) t.cuda.manual_seed_all(seed) t.cuda.manual_seed(seed) if deterministic: t.backends.cudnn.deterministic = True t.backends.cudnn.benchmark = False set_seed(0) bsuite_id = 'cartpole_swingup/0' raw_env = bsuite.load_from_id(bsuite_id) # method 1 sweep.SETTINGS[bsuite_id]['seed']=0 for episode in range(10): timestep = raw_env.reset() total_reward = 0 while not timestep.last(): action = np.random.choice(raw_env.action_spec().num_values) timestep = raw_env.step(action) total_reward += timestep.reward print(episode,total_reward) # method 2 env = gym_wrapper.GymFromDMEnv(raw_env) env.seed(seed=0) for episode in range(10): timestep = env.reset() total_reward = 0 done = False while not done: action = np.random.choice(raw_env.action_spec().num_values) sn,r,done,_ = env.step(action) total_reward += r print(episode,total_reward)
@adamxyang Did you find solution to this as I am having the same issue. I tried fixing the seed but I am still getting different results on different runs. Thanks.
from bsuite.
Related Issues (20)
- environment/experiment with continuous action space, Box() HOT 1
- Rendering control environments HOT 2
- Cannot import Random HOT 2
- bsuite_tutorial problem when build PPO OpenAI baseline agent HOT 1
- How is the 'generalization' score computed? HOT 1
- DQN mnist & mountain car performance HOT 16
- setup.py broken after last commit HOT 2
- dependency on trfl breaks TF2 HOT 2
- Question about DQN's loss HOT 1
- Using the agent's RNG, and not numpy's, to select actions HOT 1
- Importing ABC directly from collections will be removed in Python 3.10 HOT 2
- Documentation: Clarify mapping from high-level agent properties to experiments and environments HOT 2
- The signature for `update` does not allow for sarsa or n-step methods? HOT 1
- BootDQN+ not matching claimed performance HOT 17
- Cartpole environment observation parameters HOT 1
- `Catch._observation` does not follow the other environments with `_get_observation`
- How to add the results to results.py? What's the results format should be? HOT 1
- Tensorflow BOOT DQN agent loses performance after first iteration
- Incompatible with numpy>0.24
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bsuite.