Comments (21)
liuruoze
commented on May 14, 2024
File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/feature.py", line 60, in state2feature
feature_3 = map_data.reshape(batch_size, AHP.map_channels * AHP.minimap_size * AHP.minimap_size)
在这一行，你首先检验一下，batch_size=？AHP.map_channels=? AHP.minimap_size? 这个问题是由于reshape时size不匹配导致的，所以把相关数据量先打印出来，然后再找到问题原因解决
from mini-alphastar.
ZHQ-air
commented on May 14, 2024
您好，我打印了相关的参数：batch_size=1 AHP.map_channels=34 AHP.minimap_size=64
shape '[1, 139264]' is invalid for input of size 73728
意味着 batch_size*AHP.map_channels*AHP.minimap_size*AHP.minimap_size=73728 才可以吧？
那接下来我怎么操作呢？
from mini-alphastar.
ZHQ-air
commented on May 14, 2024
我可以在参数中根据自己的需求定义 重新定义 AHP.minimap_size 的值么？我其实关于这些参数是否能够根据自己的需求改动也不太清楚，主要是现在对这份代码还没有完全看懂，想先跑通再深入研究
from mini-alphastar.
ZHQ-air
commented on May 14, 2024
这里我改成 feature_3 = map_data.reshape(batch_size, map_data.shape[0] * map_data.shape[1] * map_data.shape[2]*map_data.shape[3])，解决了这个运行错误。但我再次运行代码，又出现了如下错误提示。
(base) zhq@Ubuntu20:~/Doctor/RL_Project/mini-AlphaStar$ python run.py
arch init
sl init
rl init
lt init
core init
transformer init
third init
lib init
alphastarmini init
pygame 2.0.1 (SDL 2.0.14, Python 3.8.5)
Hello from the pygame community. https://www.pygame.org/contribute.html
run init
length of replay_files: 5
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 26.53it/s]
training_size: 4
val_size: 1
torch.cuda.device_count(): 1
traj_batch.shape: torch.Size([4, 4, 185131])
Traceback (most recent call last):
File "run.py", line 46, in <module>
sl_train_by_pickle.test(on_server=P.on_server)
File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/sl_train_by_pickle.py", line 224, in test
train_for_val(replays, None, agent) # select the best hyper-parameters
File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/sl_train_by_pickle.py", line 142, in train_for_val
loss, loss_list = Loss.get_sl_loss(traj, agent.model)
File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/sl_loss.py", line 38, in get_sl_loss
label = traj_batch[:, :, feature_size:feature_size + label_size].reshape(batch_size * seq_len, label_size)
RuntimeError: shape '[16, 66302]' is invalid for input of size 12272
我只要更改reshape到合适的尺度就可以了么？但出现这种问题，我感觉是我的Replsys或Maps的问题
from mini-alphastar.
liuruoze
commented on May 14, 2024
可以把map_data的shape打印出来给我看下么？
在feature_3 = map_data.reshape(batch_size, AHP.map_channels * AHP.minimap_size * AHP.minimap_size)
这句话之前
执行
print('map_data.shape', map_data.shape)
然后同理，
也请你把lable的shape打印出来我看下
之所以我在这里没有使用自动适应的的reshape（就像你改的一样），就是为了让我的参数保持一定的敏感性，一但错了，就会报错
这个问题可能是我后面不断在调RL的时候，导致一些参数在SL那里的修改没考虑。
碰到这个问题，首先是查输入数据的pickle那里的数据的纬度是什么样的
然后再检查参数
如果pickle那里处理没问题，是需要调整参数让其匹配pickle的输入的
from mini-alphastar.
ZHQ-air
commented on May 14, 2024
map_data.shape: torch.Size([1, 18, 64, 64])
base) zhq@Ubuntu20:~/Doctor/RL_Project/mini-AlphaStar$ python run.py
arch init
sl init
rl init
lt init
core init
transformer init
third init
lib init
alphastarmini init
pygame 2.0.1 (SDL 2.0.14, Python 3.8.5)
Hello from the pygame community. https://www.pygame.org/contribute.html
run init
length of replay_files: 5
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 22.55it/s]
training_size: 4
val_size: 1
torch.cuda.device_count(): 1
map_data.shape----------------------------------------------- torch.Size([1, 18, 64, 64])
Traceback (most recent call last):
File "run.py", line 46, in <module>
sl_train_by_pickle.test(on_server=P.on_server)
File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/sl_train_by_pickle.py", line 224, in test
train_for_val(replays, None, agent) # select the best hyper-parameters
File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/sl_train_by_pickle.py", line 132, in train_for_val
traj = next(iter(train_loader)) # 通过iter()函数获取这些可迭代对象的迭代器。然后我们可以对获取到的迭代器不断使⽤next()函数来获取下⼀条数据。iter()函数实际上就是调⽤了可迭代对象的
File "/home/zhq/anaconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 517, in __next__
data = self._next_data()
File "/home/zhq/anaconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 557, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/home/zhq/anaconda3/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/zhq/anaconda3/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/dataset_pickle.py", line 110, in __getitem__
replay = self._get_random_trajectory(index)
File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/dataset_pickle.py", line 105, in _get_random_trajectory
the_item = next(iter(traj_loader))
File "/home/zhq/anaconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 517, in __next__
data = self._next_data()
File "/home/zhq/anaconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 557, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/home/zhq/anaconda3/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/zhq/anaconda3/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/dataset_pickle.py", line 73, in __getitem__
feature, label = obs2feature(obs, self.agent)
File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/dataset_pickle.py", line 31, in obs2feature
feature = Feature.state2feature(s)
File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/feature.py", line 65, in state2feature
feature_3 = map_data.reshape(batch_size, AHP.map_channels * AHP.minimap_size * AHP.minimap_size)
RuntimeError: shape '[1, 139264]' is invalid for input of size 73728
然后我将feature.py中的feature_3 = map_data.reshape(batch_size, AHP.map_channels * AHP.minimap_size * AHP.minimap_size)
改为
feature_3 = map_data.reshape(1, map_data.shape[0]*map_data.shape[1]*map_data.shape[2]*map_data.shape[3])
运行代码后打印 traj_batch.shape: torch.Size([4, 4, 185131])
代码中的 label = traj_batch[:, :, feature_size:feature_size + label_size].reshape(batch_size * seq_len, label_size)
(base) zhq@Ubuntu20:~/Doctor/RL_Project/mini-AlphaStar$ python run.py
arch init
sl init
rl init
lt init
core init
transformer init
third init
lib init
alphastarmini init
pygame 2.0.1 (SDL 2.0.14, Python 3.8.5)
Hello from the pygame community. https://www.pygame.org/contribute.html
run init
length of replay_files: 5
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 24.38it/s]
training_size: 4
val_size: 1
torch.cuda.device_count(): 1
traj_batch.shape--------------: torch.Size([4, 4, 185131])
Traceback (most recent call last):
File "run.py", line 46, in <module>
sl_train_by_pickle.test(on_server=P.on_server)
File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/sl_train_by_pickle.py", line 224, in test
train_for_val(replays, None, agent) # select the best hyper-parameters
File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/sl_train_by_pickle.py", line 142, in train_for_val
loss, loss_list = Loss.get_sl_loss(traj, agent.model)
File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/sl_loss.py", line 45, in get_sl_loss
label = traj_batch[:, :, feature_size:feature_size + label_size].reshape(batch_size * seq_len, label_size)
RuntimeError: shape '[16, 66302]' is invalid for input of size 12272
我怀疑是我们回放文件的问题，然后我把3.16.1-Pack_2中的回放文件过滤（前面的回放文件全是用的3.16.1-Pack_1中的回放文件），过滤好后运行transform_replay_data.py，却提示如下错误
(base) zhq@Ubuntu20:~/Doctor/RL_Project/mini-AlphaStar$ python run.py
arch init
sl init
rl init
lt init
core init
transformer init
third init
lib init
alphastarmini init
pygame 2.0.1 (SDL 2.0.14, Python 3.8.5)
Hello from the pygame community. https://www.pygame.org/contribute.html
run init
replay_version-----------------------------------------------: 3.16.1
REPLAY_PATH----------------------------------------------: /home/zhq/Doctor/RL_Project/mini-AlphaStar/data/filtered_replays_2/
length of replay_files-----------------------------------------------: 82
Version: B55958 (SC2.3.16)
Build: Jul 31 2017 13:19:41
Command Line: '"/home/zhq/StarCraftII/Versions/Base55958/SC2_x64" -listen 127.0.0.1 -port 18258 -dataDir /home/zhq/StarCraftII/ -tempDir /tmp/sc-7akrz589/ -dataVersion 5BD7C31B44525DAB46E64C4602A81DC2 -eglpath libEGL.so'
Starting up...
Startup Phase 1 complete
Startup Phase 2 complete
Creating stub renderer...
Listening on: 127.0.0.1:18258 (18258)
Startup Phase 3 complete. Ready for commands.
0%| | 0/82 [00:00<?, ?it/s]replay_path: /home/zhq/Doctor/RL_Project/mini-AlphaStar/data/filtered_replays_2/ffff5ca68badb2ae19559f807e8eee000df17f1f467f7218db05914b1bc179fa.SC2Replay
----------------------- Replay info ------------------------
map_name: "Acolyte LE"
player_info {
player_info {
player_id: 1
race_requested: Protoss
race_actual: Protoss
}
player_result {
player_id: 1
result: Victory
}
player_mmr: 3688
player_apm: 111
}
player_info {
player_info {
player_id: 2
race_requested: Protoss
race_actual: Protoss
}
player_result {
player_id: 2
result: Defeat
}
player_mmr: 3683
player_apm: 102
}
game_duration_loops: 20274
game_duration_seconds: 905.1524658203125
game_version: "3.16.0.55505"
data_build: 55505
base_build: 55505
data_version: "60718A7CA50D0DF42987A30CF87BCB80"
------------------------------------------------------------
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface enabled
Configure: score interface enabled
Configure: render interface disabled
Launching next game.
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Error signal: '11'
Backtrace:
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x652f73]
/lib/x86_64-linux-gnu/libc.so.6(+0x46210)[0x7f2be3ee3210]
/lib/x86_64-linux-gnu/libc.so.6(+0x18c702)[0x7f2be4029702]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x167dd7d]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x1283074]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x127c967]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x127c816]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x127c82c]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x127c82c]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x127c82c]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x127c82c]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x127c82c]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x127c82c]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x127c5cb]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x127d35e]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x12923db]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x655788]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x1b0cc79]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x1930011]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x19338e6]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x5ed4d5]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f2be3ec40b3]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x5ed2d9]
0%|
from mini-alphastar.
liuruoze
commented on May 14, 2024
应该不是回放文件的问题
我大概看明白了，还是reshap的问题
length of replay_files: 5
现在你用了5个replay文件做的测试，试下用10个3.16.1-Pack_1（先不考虑3.16.1-Pack_2的）的replay再跑一遍。
我这边也跑下实验，看看分析下原因。
from mini-alphastar.
liuruoze
commented on May 14, 2024
这是一个bug，用git下一下最新的版本就可以解决这个问题了。
简单解释一下：
主要是由map_channels和scatter_channels没有区分导致的问题。
下最新版的话，可以用git pull一下，或者直接下载main branch的zip包。直接下1.02的release包是没有效的，因为这个更新会反映在后面的1.03版才能看到
from mini-alphastar.
ZHQ-air
commented on May 14, 2024
我在transform_replay_data.py文件中更改 max_replays = 10，运行sl_train_by_pickle.py后还是出现和以前一样的错误。
但我在test_alphastar_replay.py转换replay文件的时候发现一个问题，就是我在转换pickle文件的过程中,在转换第一个和第七个文件时分别提示了如下的警告信息，不知道是不是因为replay文件转换的时候提示的这个警告信息所影响的（刚看到您上面的回复，太感谢您了）
----------------------- Replay info ------------------------
map_name: "Odyssey LE"
player_info {
player_info {
player_id: 1
race_requested: Protoss
race_actual: Protoss
}
player_result {
player_id: 1
result: Defeat
}
player_mmr: 4166
player_apm: 230
}
player_info {
player_info {
player_id: 2
race_requested: Protoss
race_actual: Protoss
}
player_result {
player_id: 2
result: Victory
}
player_mmr: 4115
player_apm: 141
}
game_duration_loops: 15042
game_duration_seconds: 671.564697265625
game_version: "3.16.1.55958"
data_build: 55958
base_build: 55958
data_version: "5BD7C31B44525DAB46E64C4602A81DC2"
------------------------------------------------------------
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface enabled
Configure: score interface enabled
Configure: render interface disabled
Launching next game.
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Starting replay 'TempStartReplay.SC2Replay'
Game has started.
**WARNING:absl:Not found tag! 4358668289**
max frames test, break out!
begin save!
end save!
1%|█▉ | 1/72 [00:38<45:13, 38.22s/it]replay_path: /home/zhq/Doctor/RL_Project/mini-AlphaStar/data/filtered_replays_1/fffeb9b9437fde426b5e51a700bd339e0eedb127a2106ff9920b65c2bccfd4f4.SC2Replay
----------------------- Replay info ------------------------
map_name: "Acolyte LE"
player_info {
player_info {
player_id: 1
race_requested: Protoss
race_actual: Protoss
}
player_result {
player_id: 1
result: Defeat
}
player_mmr: 4024
player_apm: 88
}
player_info {
player_info {
player_id: 2
race_requested: Protoss
race_actual: Protoss
}
player_result {
player_id: 2
result: Victory
}
player_mmr: 4039
player_apm: 102
}
game_duration_loops: 69901
game_duration_seconds: 3120.798095703125
game_version: "3.16.1.55958"
data_build: 55958
base_build: 55958
data_version: "5BD7C31B44525DAB46E64C4602A81DC2"
------------------------------------------------------------
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface enabled
Configure: score interface enabled
Configure: render interface disabled
Launching next game.
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Starting replay 'TempStartReplay.SC2Replay'
Game has started.
`WARNING:absl:Unknown ability_id: 1832. This is probably dance or cheer, or some unknown new or map specific ability. Treating it as a no-op.`
max frames test, break out!
begin save!
end save!
max replays test, break out!
12%|█████████████████▍
from mini-alphastar.
liuruoze
commented on May 14, 2024
WARNING:absl:Unknown ability_id: 1832. This is probably dance or cheer, or some unknown new or map specific ability. Treating it as a no-op.
这个警告是没有问题的。PySC2会自动把这个ability_id转换成no_op。这个ability_id往往是由于SC2更新新版本（增加了一些可能的动作），而PySC2没有感知到的原因。一般来说，这些ability_id数量很少，对agent的性能影响也不大。
from mini-alphastar.
ZHQ-air
commented on May 14, 2024
我是知道进行强化学习前要进行监督学习的，但我想看下直接用强化学习进行训练的效果。于是我直接运行
from alphastarmini.core.rl import rl_train_with_replay
rl_train_with_replay.test(on_server=P.on_server, replay_path=P.replay_path)
会提示如下错误：
(base) zhq@Ubuntu20:~/Doctor/RL_Project/mini-AlphaStar$ python run.py
arch init
sl init
rl init
lt init
core init
transformer init
third init
lib init
alphastarmini init
pygame 2.0.1 (SDL 2.0.14, Python 3.8.5)
Hello from the pygame community. https://www.pygame.org/contribute.html
run init
No models are found!
No models are found!
learner trajectories size: 0
learner trajectories size: 0
start_time before training: 2021-09-10 14:31:42
map name: AbyssalReef
player.name: MainPlayer
opponent.name: Historical
player.race: Race.protoss
opponent.race: Race.protoss
Version: B55958 (SC2.3.16)
Build: Jul 31 2017 13:19:41
Command Line: '"/home/zhq/StarCraftII/Versions/Base55958/SC2_x64" -listen 127.0.0.1 -port 23022 -dataDir /home/zhq/StarCraftII/ -tempDir /tmp/sc-jlvl3_s5/'
Starting up...
Startup Phase 1 complete
learner trajectories size: 0
Startup Phase 2 complete
Creating stub renderer...
Listening on: 127.0.0.1:23022 (23022)
Startup Phase 3 complete. Ready for commands.
learner trajectories size: 0
Version: B55958 (SC2.3.16)
Build: Jul 31 2017 13:19:41
Command Line: '"/home/zhq/StarCraftII/Versions/Base55958/SC2_x64" -listen 127.0.0.1 -port 20923 -dataDir /home/zhq/StarCraftII/ -tempDir /tmp/sc-pa37zt2s/'
Starting up...
Startup Phase 1 complete
learner trajectories size: 0
Startup Phase 2 complete
Creating stub renderer...
Listening on: 127.0.0.1:20923 (20923)
Startup Phase 3 complete. Ready for commands.
learner trajectories size: 0
Unable to join game.
Unable to join game.
ActorLoop.run() Exception cause return, Detials of the Exception: SC2APIProtocol.ResponseJoinGame.Error.FeatureUnsupported: 'Feature layers unsupported for multiplayer'
Traceback (most recent call last):
File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/rl/actor_plus_z.py", line 119, in run
with self.create_env(self.player, self.opponent) as env:
File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/rl/actor_plus_z.py", line 400, in create_env
env = SC2Env(map_name=map_name,
File "/home/zhq/anaconda3/lib/python3.8/site-packages/pysc2/env/sc2_env.py", line 288, in __init__
self._create_join()
File "/home/zhq/anaconda3/lib/python3.8/site-packages/pysc2/env/sc2_env.py", line 434, in _create_join
self._parallel.run((c.join_game, join)
File "/home/zhq/anaconda3/lib/python3.8/site-packages/pysc2/lib/run_parallel.py", line 73, in run
raise f.exception()
File "/home/zhq/anaconda3/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/zhq/anaconda3/lib/python3.8/site-packages/pysc2/lib/remote_controller.py", line 99, in _valid_status
return func(self, *args, **kwargs)
File "/home/zhq/anaconda3/lib/python3.8/site-packages/pysc2/lib/remote_controller.py", line 74, in _check_error
return check_error(func(*args, **kwargs), error_enum)
File "/home/zhq/anaconda3/lib/python3.8/site-packages/pysc2/lib/remote_controller.py", line 65, in check_error
raise RequestError("%s.%s: '%s'" % (enum_name, error_name, details), res)
pysc2.lib.remote_controller.RequestError: SC2APIProtocol.ResponseJoinGame.Error.FeatureUnsupported: 'Feature layers unsupported for multiplayer'
Actor stops!
Learner also stops!
run over
RequestQuit command received.
Closing Application...
RequestQuit command received.
Closing Application...
出错的原因是我没有首先进行监督学习训练，直接进行强化学习训练的原因导致的么？
from mini-alphastar.
liuruoze
commented on May 14, 2024
强化学习的错误，是因为你用的是3.16.1版本
注意信息中的这句话：
Version: B55958 (SC2.3.16)
而SC2的3.16.1版本是不支持两个agent互相对战的（只能一个agent打另一个电脑），所以报错了。解决的办法是要切换到4.0版本以上，例如用4.10的版本。
所以你在服务器上要常备两个SC2的版本，比方说在你的home目录下，应该有下面的结构：
StarCraftII
StarCraftII-3.16.1
这时候StarCraftII目录里应该是4.10的版本，这时候做强化学习；
当要做监督学习，并且需要训练3.16.1的replay时，把两个目录的名字改下，变成下面的结构：
StarCraftII-4.10
StarCraftII
看出来了吗？SC2找到的就是哪个文件夹叫“StarCraftII”，所以你要用哪个版本时，就把那个版本的名字的后缀去掉，其他版本保留后缀，这样就能找到了。
我的做法是用一个叫StarCraftII的软链接，根据需要指向特定的版本。
这个确实比较麻烦，不过这个做法可以保证训练不出问题。StarCraft2由于版本众多，确实导致了训练不是很方便。
from mini-alphastar.
liuruoze
commented on May 14, 2024
还有，你刚才为什么只转换了5个文件的原因找到了。
你需要打开param.py，把其中的一个值改下：
on_server = False
on_server = True
on_server 要改为True，这样在transform_replay_data下才不会执行以下的逻辑：
#line 298
else:
REPLAY_PATH = "data/Replays/filtered_replays_1/"
COPY_PATH = None
SAVE_PATH = "./result.csv"
max_steps_of_replay = 120 * 22.4 # 60 * 60 * 22.4
max_replays = 5 # 1
改成True以后，会执行下面的语句：
#line 292
if on_server:
REPLAY_PATH = P.replay_path # "/home/liuruoze/data4/mini-AlphaStar/data/filtered_replays_1/"
COPY_PATH = None
SAVE_PATH = "./result.csv"
max_steps_of_replay = FLAGS.max_steps_of_replay
max_replays = FLAGS.max_replays
注意把这句话”REPLAY_PATH = P.replay_path“改成你的replay存放的路径。
其它的一些”FLAGS.“的值你可以在transform_replay_data文件的上面一些行找到（你可以根据自己需要修改）。
我这样做的原因是：当我在自己电脑（笔记本上）修改时，我把on_server 改成False，这样相关一些值取得较小，方便调试。
当我把代码传到服务器上时，就用服务器上的param.py（我在传代码的时候，我只传alphastarmini这个文件夹，param.py文件不传，run.py看情况），而服务器上的param.py种的on_server已经是True，这样直接就可以用服务器上预设好的replay_path（在不同的服务器上的replay_path的位置可能不一样，所以可以在param.py上配置好，这样以后就不需要修改了，这样子方便同时在两个以上的服务器上调试代码）
from mini-alphastar.
ZHQ-air
commented on May 14, 2024
Thank you very very much !
我现在home目录下的结构为
StarCraftII
StarCraftII-3.16.1
其中StarCraftII的版本为4.10，我直接把3.16.1的回放文夹放进去试了下，然后直接运行强化学习的代码，出现了如下错误
(base) zhq@Ubuntu20:~/Doctor/RL_Project/mini-AlphaStar$ python run.py
arch init
sl init
rl init
lt init
core init
transformer init
third init
lib init
alphastarmini init
pygame 2.0.1 (SDL 2.0.14, Python 3.8.5)
Hello from the pygame community. https://www.pygame.org/contribute.html
run init
No models are found!
No models are found!
learner trajectories size: 0
learner trajectories size: 0
start_time before training: 2021-09-10 17:15:37
ActorLoop.run() Exception cause return, Detials of the Exception: Unknown game version: 3.16.1. Known versions: ['4.10.0', 'latest'].
Traceback (most recent call last):
File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/rl/actor_plus_z.py", line 117, in run
run_config = run_configs.get(version=self.replay_version) # the replays released by blizzard are all 3.16.1 version
File "/home/zhq/anaconda3/lib/python3.8/site-packages/pysc2/run_configs/__init__.py", line 40, in get
return max(configs.values(), key=lambda c: c.priority())(version=version)
File "/home/zhq/anaconda3/lib/python3.8/site-packages/pysc2/run_configs/platforms.py", line 179, in __init__
super(Linux, self).__init__(base_dir, "SC2_x64", version=version, env=env)
File "/home/zhq/anaconda3/lib/python3.8/site-packages/pysc2/run_configs/platforms.py", line 61, in __init__
super(LocalBase, self).__init__(
File "/home/zhq/anaconda3/lib/python3.8/site-packages/pysc2/run_configs/lib.py", line 112, in __init__
self.version = self._get_version(version)
File "/home/zhq/anaconda3/lib/python3.8/site-packages/pysc2/run_configs/lib.py", line 220, in _get_version
versions = self.get_versions(containing=game_version)
File "/home/zhq/anaconda3/lib/python3.8/site-packages/pysc2/run_configs/platforms.py", line 108, in get_versions
raise ValueError("Unknown game version: %s. Known versions: %s." % (
ValueError: Unknown game version: 3.16.1. Known versions: ['4.10.0', 'latest'].
我在想，如果把4.10版本的replay放进去的话，跟监督学习训练的replay版本也不一致啊，这里还是不太理解应该怎么操作
from mini-alphastar.
liuruoze
commented on May 14, 2024
不好意思，我的。我把一个之前的做法告诉了你。事实上，已经有更好的一个做法。
我之前是采用这种替换名字的方式来做到SC2版本的切换的。
但是这种模式在RL训练的时候用不了。因为要想与agent对战，必须使用4.0以上的版本。而同时想获得z奖励的话，必须打开一个replay（目前只有3.16.1版本的）。因此，需要同时打开4.0以上版本的星际2（要开两个窗口，一个代表训练的智能体，一个代表作为对手的智能体），以及一个3.16.1的窗口（用来跑replay）
这种做法是可以实现的：
在StarCraftII目录下，有一个Versions文件夹。里面存放了历史的SC2的各个版本
例如3.16.1的版本就叫如下的名字（55958是3.16.1的开发版本号，这个版本号可以在PySC2的run_configs/lib.py中查到）：
Base55958
如果你现在运行的SC2的4.0以上的版本，并且你的Versions文件夹有Base55958（这个文件夹你可以从3.16.1的星际2下的Versions文件夹中拷贝一个过来）。
那么通过在run_config = run_configs.get（version=‘’）
这里指定版本号就可以同时运行不同版本的星际2.
version=‘3.16.1’，运行3.16.1版本
version=‘4.1.2'，运行4.1.2版本（要有Base60321）
version=‘4.10.0'，运行4.10版本（要有Base75689）
version=None，运行最新的版本
所以，你现在要做的就是把当前的StarCraftII设为4.10的那个文件夹，然后再把Base55958这个文件夹从3.16.1的星际2下的Versions文件夹下面拷贝一份到这个文件夹下的Versions文件夹下面。
之后再跑RL代码，就可以正确运行了。
from mini-alphastar.
ZHQ-air
commented on May 14, 2024
您好，我现在把3.16.1的星际2下的Versions文件夹下面拷贝一份到了4.10.0版本的StarCraftII目录下的Versions文件夹下面了，但是进行RL训练时，提示如下错误：
(base) zhq@Ubuntu20:~/Doctor/RL_Project/mini-AlphaStar$ python run.py
arch init
sl init
rl init
lt init
core init
transformer init
third init
lib init
alphastarmini init
pygame 2.0.1 (SDL 2.0.14, Python 3.8.5)
Hello from the pygame community. https://www.pygame.org/contribute.html
run init
No models are found!
No models are found!
learner trajectories size: 0
learner trajectories size: 0
start_time before training: 2021-09-13 15:32:32
map name: AbyssalReef
player.name: MainPlayer
opponent.name: Historical
player.race: Race.protoss
opponent.race: Race.protoss
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Command Line: '"/home/zhq/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 15456 -dataDir /home/zhq/StarCraftII/ -tempDir /tmp/sc-03ngqdlw/'
Starting up...
Startup Phase 1 complete
learner trajectories size: 0
Startup Phase 2 complete
Creating stub renderer...
Listening on: 127.0.0.1:15456
Startup Phase 3 complete. Ready for commands.
learner trajectories size: 0
ConnectHandler: Request from 127.0.0.1:33446 accepted
ReadyHandler: 127.0.0.1:33446 ready
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Command Line: '"/home/zhq/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 22060 -dataDir /home/zhq/StarCraftII/ -tempDir /tmp/sc-r8wvxl1l/'
Starting up...
Startup Phase 1 complete
learner trajectories size: 0
Startup Phase 2 complete
Creating stub renderer...
Listening on: 127.0.0.1:22060
Startup Phase 3 complete. Ready for commands.
learner trajectories size: 0
ConnectHandler: Request from 127.0.0.1:33588 accepted
ReadyHandler: 127.0.0.1:33588 ready
Requesting to join a multiplayer game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface enabled
Configure: score interface enabled
Configure: render interface disabled
Create game with map path: AbyssalReefLE.SC2Map
Requesting to join a multiplayer game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface enabled
Configure: score interface enabled
Configure: render interface disabled
learner trajectories size: 0
learner trajectories size: 0
Attempting to join a game with the map: Ladder2017Season4/AbyssalReefLE.SC2Map
learner trajectories size: 0
learner trajectories size: 0
learner trajectories size: 0
learner trajectories size: 0
learner trajectories size: 0
learner trajectories size: 0
learner trajectories size: 0
Game has started.
Using default stable ids, none found at: /home/zhq/StarCraftII/stableid.json
Successfully loaded stable ids: GameData\stableid.json
Sending ResponseJoinGame
Game has started.
Using default stable ids, none found at: /home/zhq/StarCraftII/stableid.json
Successfully loaded stable ids: GameData\stableid.json
Sending ResponseJoinGame
teacher: <alphastarmini.core.rl.alphastar_agent.AlphaStarAgent object at 0x7f8567ec5dc0>
start_time before reset: 2021-09-13 15:32:45
total_episodes: 1
Version: B55958 (SC2.3.16)
Build: Jul 31 2017 13:19:41
Command Line: '"/home/zhq/StarCraftII/Versions/Base55958/SC2_x64" -listen 127.0.0.1 -port 24299 -dataDir /home/zhq/StarCraftII/ -tempDir /tmp/sc-iyia_tax/ -dataVersion 5BD7C31B44525DAB46E64C4602A81DC2 -eglpath libEGL.so'
Starting up...
Startup Phase 1 complete
learner trajectories size: 0
Fatal error encountered!
Backtrace:
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x653068]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x653aa0]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x6534f2]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x5f0ac8]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x19348d5]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x5ed44d]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f1dafade0b3]
/home/zhq/StarCraftII/Versions/Base55958/SC2_x64[0x5ed2d9]
Error condition code: '9'
Error suffix: 'Error'
Error preface: 'A fatal error occurred:'
Error failure: 'e_errorIdStreaming'
Error unique: '(master index)'
Error details: '"AppErrors.txt"'
Error description: 'AppErrors.txt'
learner trajectories size: 0
WARNING:absl:SC2 isn't running, so bailing early on the websocket connection.
RequestQuit command received.
Closing Application...
DataHandler: unable to parse websocket frame.
CloseHandler: 127.0.0.1:33446 disconnected
DataHandler: unable to parse websocket frame.
RequestQuit command received.
CloseHandler: 127.0.0.1:33588 disconnected
Closing Application...
ResponseThread: No connection, dropping the response.
ActorLoop.run() Exception cause return, Detials of the Exception: Failed to connect to the SC2 websocket. Is it up?
Traceback (most recent call last):
File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/rl/actor_plus_z.py", line 152, in run
with run_config.start(full_screen=False) as controller:
File "/home/zhq/anaconda3/lib/python3.8/site-packages/pysc2/run_configs/platforms.py", line 204, in start
return super(Linux, self).start(
File "/home/zhq/anaconda3/lib/python3.8/site-packages/pysc2/run_configs/platforms.py", line 87, in start
return sc_process.StarcraftProcess(
File "/home/zhq/anaconda3/lib/python3.8/site-packages/pysc2/lib/sc_process.py", line 142, in __init__
self._controller = remote_controller.RemoteController(
File "/home/zhq/anaconda3/lib/python3.8/site-packages/pysc2/lib/remote_controller.py", line 146, in __init__
sock = self._connect(host, port, proc, timeout_seconds)
File "/home/zhq/anaconda3/lib/python3.8/site-packages/pysc2/lib/stopwatch.py", line 212, in _stopwatch
return func(*args, **kwargs)
File "/home/zhq/anaconda3/lib/python3.8/site-packages/pysc2/lib/remote_controller.py", line 180, in _connect
raise ConnectError("Failed to connect to the SC2 websocket. Is it up?")
pysc2.lib.remote_controller.ConnectError: Failed to connect to the SC2 websocket. Is it up?
Actor stops!
Learner also stops!
run over
from mini-alphastar.
liuruoze
commented on May 14, 2024
好像是有点问题，貌似不能直接拷贝？具体原因我也不清楚。
但有一个方法是肯定可行的
用下面的url的版本
https://colab.research.google.com/drive/1TzO2Wi9KLjfBZeOqjGgjlxmwU5IX4wIV#scrollTo=XoZu1wZWLrfP
注意看
creating: /root/StarCraftII/Versions/Shaders41359/
creating: /root/StarCraftII/Versions/Shaders41359/OpenGL2/
inflating: /root/StarCraftII/Versions/Shaders41359/OpenGL2/baselineCache.bin
creating: /root/StarCraftII/Versions/Base60321/
inflating: /root/StarCraftII/Versions/Base60321/SC2_x64
creating: /root/StarCraftII/Versions/Base59587/
inflating: /root/StarCraftII/Versions/Base59587/SC2_x64
creating: /root/StarCraftII/Versions/Base55958/
inflating: /root/StarCraftII/Versions/Base55958/SC2_x64
creating: /root/StarCraftII/Versions/Base57507/
inflating: /root/StarCraftII/Versions/Base57507/SC2_x64
creating: /root/StarCraftII/Versions/Shaders41358/
creating: /root/StarCraftII/Versions/Shaders41358/OpenGL2/
inflating: /root/StarCraftII/Versions/Shaders41358/OpenGL2/baselineCache.bin
creating: /root/StarCraftII/Versions/Base56787/
inflating: /root/StarCraftII/Versions/Base56787/SC2_x64
creating: /root/StarCraftII/Versions/Base58400/
inflating: /root/StarCraftII/Versions/Base58400/SC2_x64
这个是一个4.1.2版本的Linux SC2。但它下面有Base55958也就是3.16.1的进程
用这个作为主SC2文件夹，RL的两个进程用4.1.2跑，replay会自动用Versions/Base55958下的文件来跑
至于怎么用4.10的版本，也许可以试试把4.10的Base文件夹放到这个Versions下面？（不保证一定能work）
from mini-alphastar.
ZHQ-air
commented on May 14, 2024
用https://colab.research.google.com/drive/1TzO2Wi9KLjfBZeOqjGgjlxmwU5IX4wIV#scrollTo=XoZu1wZWLrfP
能够进行RL的训练了，感谢您的指导！
不知道我们有SC2 AI 的交流群或论坛么？想以后跟踪贵实验室的工作动态
from mini-alphastar.
liuruoze
commented on May 14, 2024
目前还没有，不过我们之后可以建一个 :)
from mini-alphastar.
ZHQ-air
commented on May 14, 2024
太好了，赞
from mini-alphastar.
liuruoze
commented on May 14, 2024
你可以发一个邮件到我的邮箱，这样我可以把我的手机号通过邮件发给你，然后你可以用微信查找到我的账号，然后我拉一个群。
from mini-alphastar.
Related Issues (20)
- mini-AlphaStar V1.05编译报错 HOT 12
- 运行scalar_encoder.test()结果输出非法值 HOT 2
- CUDA out of memory HOT 4
- Starcraft 2 version problem HOT 10
- Reproducing the recent SL model featured in README.md HOT 12
- TypeError HOT 7
- Multi node training HOT 2
- Unable to Process Replays? HOT 7
- Trained model replays. HOT 2
- What is the meaning of Features and Labels? HOT 2
- 运行run.py时遇到的报错 HOT 7
- Issue with result reproduction HOT 3
- Unable to process action with ability_id: xxxx HOT 2
- 想请教一下关于预训练的数据问题 HOT 10
- Reproducing results on latest Starcraft version (5.0.9) HOT 1
- Is it possible to share some experiment details about model training ? HOT 1
- Some questions in Rl-training HOT 1
- 如何得到训练结果的GIF
- 运行rl_vs_inner_bot_mp.py没有输出
- 如何得到训练rl模型对内置ai的胜率 HOT 15
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mini-alphastar.