mrwangyou / dbrl

A Gym Dogfighting Simulation Benchmark for Reinforcement Learning Research

• 👋 Hi, I’m @mrwangyou
• 👀 I’m interested in air combat simulation
• 🌱 I’m currently learning reinforcement learning
• 💞️ I’m looking to collaborate on dogfight benchmark
• 📫 Mail [email protected]

dbrl's Issues

dogfight无法正常获取飞机发射导弹的状态

你好，在运行dogfight DBRL或者client example missile custom physics.py时，执行到get_missile_state(missile_id),会出现socket错误无法获取该导弹的状态，显示‘Nonetype’object has no attribute 'decode', 我debug 到dogfight_client.py对应函数时，missile_id是有效的str，但就是无法返回它的states. 请问你知道咋回事吗

在使用过程中的问题

您好，想问一下在使用中出现问题
Could not open file: Path "D:/Anaconda/install/envs/DBRL/lib/site-packages/jsbsim/./data_output/flightgear1.xml"

flightgearx.xml渲染文件找不到

请问('./data_output/flightgear1.xml')文件是怎么定义的呢？代码中没有体现

您好，请问您使用的仿真平台是否支持修改想定？如将现有的2v2扩充到6v6，12v12等，来进行强化学习算法的训练？

你好，我想在GPU上运行这些代码，应该怎么去设置呢？

自我博弈訓練

你好，請問有可能在這上面進行自我博弈訓練嗎

是否可以接入其他模型接口如PPO，TD3，DQN来进行训练？

开发者您好，我想要引入其他模型来进行对比试验，本项目有没有使用其他模型进行训练的功能呢

请问如果通过2个FlightGear进行战斗？

通过FlightGear命令起动了两个FlightGear：
fgfs --fdm=null --native-fdm=socket,in,60,,5550,udp --multiplay=out,10,127.0.0.1,5000 --multiplay=in,10,127.0.0.1,5001 --callsign=Test1

fgfs --fdm=null --native-fdm=socket,in,60,,5550,udp --multiplay=out,10,127.0.0.1,5001 --multiplay=in,10,127.0.0.1,5000 --callsign=Test2

请问如何通过代码开启两个战斗机进行混战模拟？

你好，有在dogfight-sandbox-hg2上进行1v1或者2v2空战对抗的示例么？

dogfightenv环境sac_df训练问题

您好，我刚刚开始尝试在dogfightenv上训练，请问直接运行sac_df，使用stable_baselines3的算法能达到什么效果，timesteps设置为多少reward能够收敛呢。我初步尝试了一下，没有做任何修改，模型似乎没有收敛。

gym需要更新为gymnasium了

gym的API变成gymnasium了，建议进行更新

场景Replay相关问题

作者你好！非常感谢你的分享。关于场景的回放我有一些疑惑。

• 把render mode打开的话，训练开始后，dogfight软件界面内的视线会一直锁定在不飞的飞机上，反而是会另外生成一架飞机并进行起飞，但是现在我想把视角转移到实际飞行并训练的那架飞机上，观看他的飞行姿态，这个有办法实现吗？
• 我观察到目前的gym环境采用的dogfight版本为1.0.2，dogfight在后续版本里给出了Recorder API接口，我试着用Recorder去实现第一点里我想要的效果，但是没能成功，在Recorder模式下我的视角被锁定在了一架航母上。请问你有使用过后续版本的Recorder API吗？

a bug while training dogfight algorithm

I follow the steps in the Tutorial to start my training. Initially, everything seemed to be alright, but later, the training process was terminated and raise a bug:

Traceback (most recent call last):

File "XXX\anaconda\envs\JSB_DF_GYM\share\JSBSim\DBRL-main\src\models\sac_jsbsim.py", line 49, in <module> 
    model.learn(total_timesteps=10000000, log_interval=1) 
File "XXX\anaconda\envs\jsb_df_gym\lib\site-packages\stable_baselines3\sac\sac.py", line 309, in learn
    return super().learn( 
File "XXX\anaconda\envs\jsb_df_gym\lib\site-packages\stable_baselines3\common\off_policy_algorithm.py", line 356, in learn
    rollout = self.collect_rollouts(  
File "XXX\anaconda\envs\jsb_df_gym\lib\site-packages\stable_baselines3\common\off_policy_algorithm.py", line 586, in collect_rollouts 
    actions, buffer_actions = self._sample_action(learning_starts, action_noise, env.num_envs)
File "XXX\anaconda\envs\jsb_df_gym\lib\site-packages\stable_baselines3\common\off_policy_algorithm.py", line 417, in _sample_action
    unscaled_action, _ = self.predict(self._last_obs, deterministic=False)
File "XXX\anaconda\envs\jsb_df_gym\lib\site-packages\stable_baselines3\common\base_class.py", line 632, in predict
    return self.policy.predict(observation, state, episode_start, deterministic)  
File "XXX\anaconda\envs\jsb_df_gym\lib\site-packages\stable_baselines3\common\policies.py", line 336, in predict
    actions = self._predict(observation, deterministic=deterministic) 
File "XXX\anaconda\envs\jsb_df_gym\lib\site-packages\stable_baselines3\sac\policies.py", line 356, in _predict
    return self.actor(observation, deterministic) 
File "XXX\anaconda\envs\jsb_df_gym\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs) 
File "XXX\anaconda\envs\jsb_df_gym\lib\site-packages\stable_baselines3\sac\policies.py", line 177, in forward
    return self.action_dist.actions_from_params(mean_actions, log_std, deterministic=deterministic, **kwargs) 
File "XXX\anaconda\envs\jsb_df_gym\lib\site-packages\stable_baselines3\common\distributions.py", line 179, in actions_from_params
    self.proba_distribution(mean_actions, log_std)
File "XXX\anaconda\envs\jsb_df_gym\lib\site-packages\stable_baselines3\common\distributions.py", line 211, in proba_distribution
    super().proba_distribution(mean_actions, log_std) 
File "XXX\anaconda\envs\jsb_df_gym\lib\site-packages\stable_baselines3\common\distributions.py", line 153, in proba_distribution
    self.distribution = Normal(mean_actions, action_std)  
File "XXX\anaconda\envs\jsb_df_gym\lib\site-packages\torch\distributions\normal.py", line 54, in __init__
    super(Normal, self).__init__(batch_shape, validate_args=validate_args)
File "XXX\anaconda\envs\jsb_df_gym\lib\site-packages\torch\distributions\distribution.py", line 55, in __init__
    raise ValueError(   
ValueError: Expected parameter loc (Tensor of shape (1, 4)) of distribution Normal(loc: torch.Size([1, 4]), scale: torch.Size([1, 4])) to satisfy the constraint Real(), but found invalid values:  
tensor([[nan, nan, nan, nan]], device='cuda:0')

This problem recurs steadily every training session. So what is the potential cause of this bug? How should I circumvent this problem?

请问一下jsbsim.FGFDMExec的properties属性值有哪些是在哪里能够找到呢，方便告知吗。找遍了jsbsim的官网也没发现

您好，请问您的1v1 baseline模型训练的情况如何

代码运行遇到了一个问题，请问该如何解决

1.运行第一个example时报错PermissionError: [Errno 13] Permission denied: 'log\sac_jsbsim'；