Comments (1)
Hello @akmandor.
As for GAIL: observations space can be both a 1D array or a 3D array and action space is a 1D array. In our implementation, observations space is not a dict, but we have taken this into account and included it in the roadmap. For now it can be a numpy array or a tensor.
For a quick example you can run the following commands:
cd dizoo/box2d/lunarlander/config
ding -m serial -c lunarlander_dqn_config.py # train lunarlander expert DQN
python lunarlander_dqn_gail_config.py # collect expert data and train lunarlander GAIL
BC is still under development and you can follow its progress in this PR: #110
Note that we have not tested GAIL with 3D observations yet, but we have extensively tested it on Mujoco and Box2D envs. If you would like to write your own 3D encoder for GAIL, you can replace it with our encoder here: https://github.com/opendilab/DI-engine/blob/main/ding/reward_model/gail_irl_model.py#L75
If you need to write an encoder that supports dict observations, you can modify our encoder in a way similar to this according to the design of your environment:
def forward(self, x) -> torch.Tensor:
x1, nx = x[‘obs1’], x[‘obs_n’]
x1, xn = preprocess(x1, xn)
out = torch.Tensor(np.concat(x1, nx))
out = self.l1(out)
out = self.a1(out)
out = self.l2(out)
out = self.a2(out)
return out
from di-engine.
Related Issues (20)
- H-PPO算法运行失败 HOT 7
- 尝试使用自定义环境出现问题 HOT 2
- gym soccer是否有文档? 其参数设置以及action的类型该如何写 HOT 3
- record a video HOT 2
- Implementation of Mean-Field MARL algorithm HOT 3
- FQF logit computation HOT 3
- 混合动作空间环境,PPO使用gae_estimator报错 HOT 3
- 如何获取每个episode的reward值 HOT 1
- TD3应用混合动作空间报错,AssertionError HOT 1
- how to get the ckpt file? HOT 2
- get "TypeError: __init__() got an unexpected keyword argument 'agent_obs_shape'" when running " python3 -u smac_5m6m_masac_config.py" HOT 2
- question for SMAC HOT 3
- docker内运行lunarlander_dqn_deploy失败 HOT 6
- cannot run GTrXL demo since v0.5.0 HOT 1
- BrokenPipeError: [WinError 232] 管道正在被关闭 has occurred, when running MARL algorithm QMIX in pettingzoo
- bug when running MARL algorithm Qmix in pettingzoo HOT 3
- gym_anytrading : could not broadcast input array from shape (62,) into shape (20,3) Please help!! HOT 3
- 马里奥代码咨询 HOT 1
- No example for Trading Environments HOT 1
- KeyError: 'obs_shape' HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from di-engine.