liuruoze / mini-alphastar Goto Github PK

(JAIR'2022) A mini-scale reproduction code of the AlphaStar program. Note: the original AlphaStar is the AI proposed by DeepMind to play StarCraft II. JAIR = Journal of Artificial Intelligence Research.

License: Apache License 2.0

Python 100.00%

deep-learning deep-neural-networks deep-reinforcement-learning gaming imitation-learning mini-alphastar multi-agent-systems pytorch reinforcement-learning sc2replay self-playing-bot starcraft-ii-bot starcraft2 starcraft2-ai supervised-learning

mini-alphastar's Introduction

I am a Ph.D. of Nanjing University (NJU) in China. Before that, I was a data scientist at CICDI China. I have led > 5 machine learning open-source projects, receiving > 6000 stars and > 2500 forks. I specialize in building deep reinforcement learning agents with applications in games (such as StarCraft I and II) and computer vision systems with applications in plate license recognition.

🔭 I focus on:

Hierarchical Reinforcement Learning topics to solve complex games or environments.
Model-based Reinforcement Learning topics to do efficient training.

😄 I am open to:

Paper review
Internship: Reinforcement Learning on Game AI.

mini-alphastar's People

Contributors

Stargazers

Watchers

Forkers

dachidahu wwxfromtju kornbergfresnel guyskapen zhq-air cliffwb wuyadie jkchromy cloneniu harryhsing yimingzhang521 cjalchange shiyuzh2007 mutnpc cainiaoluo igabriel85 helloxms findlamp respawn-id purpleyoung cozydoomer liyi593730139 rainwangphy error-1324 rahatsantosh i-am-tc athrunsunny zhaoyangbu dreamersurpass binny1446 soygema knowledgecluster qiaoptdun nwpu-ssz jmsm1g19 shuaili-xy cobase2010 nevercast marktrovinger samuel318 klonggan chikulis monsterspirit adi-vc simrit1 ruihan0495 xinzhang1999 soulkeee ng-cho-yin wenzhe-li wacoder apexsoftsv mjones00 magiccoder007 ai8868 lizhongyi123

mini-alphastar's Issues

Multi node training

Hello

Do you plan to make it possible to train agent on multiple nodes? I mean on computers that are divided by network. Maybe in form of a league like in original DeepMind paper

回放文件Replays

   您好，关于“data/Replays/filtered_replays_1”中的回放文件是怎么得到的呀？ 我尝试的操作是：

利用sc2client-proto/samples/replay-api/download_replays.py下载与我的StarCraftII（4.9.3）对应版本的回放文件，
但是运行download_replays.py文件时提示“Failed to get oauth access token.”。
其中我的运行命令是 python download_replays.py --key=我的battle.net邮箱账号 --secret=我的battle.net账号密码 --version=4.9.3 --replays_dir=/home/zhq/Doctor/RL_Project/s2client-proto/samples/replay-api/replays --extract。
请问下，这种情况是怎么回事呀？

TypeError

In run.py, I run codes below:

# 6. we use SL model and replays to do reinforcement learning

from alphastarmini.core.rl import rl_train_with_replay
rl_train_with_replay.test(on_server=P.on_server, replay_path=P.replay_path)

And the outcome is shown below with error:

run over

Please help me, thanks!!

IndexError

It is very exciting to see the implementation of Alphastar after 2 years since release of the paper. I am trying to run miniAlphaStar. But after a few tries I am stuck at an error. Could you give me some help?

Steps I take:

install the environment with "conda create -n th_1_5 python=3.7 pytorch=1.5 -c pytorch", "conda activate th_1_5" and "pip install -r requirements.txt". conda version 4.8.2.
download StarCraft from "http://blzdistsc2-a.akamaihd.net/Linux/SC2.4.6.0.67926.zip" in page "https://github.com/Blizzard/s2client-proto" and unzip to ~/
download all maps from "https://github.com/Blizzard/s2client-proto" and move them to ~/StarCraftII/Maps/
download replays with command "python download_replays.py --key=mykey --secret=mysecrete --version="4.6.0" --download_dir=./download/download-4-6 --extract --replays_dir=./replay/replay-4-6 --filter_version="delete"
make a soft link "data/Replays/filtered_replays_1/" to "replay/replay-4-6"
change line 71 of mini-AlphaStar/alphastarmini/core/sl/transform_replay_data.py from "flags.DEFINE_string("replay_version", "3.16.1", "the replays released by blizzard are all 3.16.1 version")" to "flags.DEFINE_string("replay_version", "4.6.0", "the replays released by blizzard are all 4.1.2 version")"
change "flags.DEFINE_string("no_server_replay_path", "D:/work/gitproject/mini-AlphaStar/data/Replays/small_simple64_replays/", "path of replay data")" to "flags.DEFINE_string("no_server_replay_path", "/home/user/Project/mini-AlphaStar/scripts/download_replay/third/replay/replay-4-6/", "path of replay data")"

then "python run.py"

but I got following error:

  0%|                                     | 10/60474 [00:29<19:04:31,  1.14s/it]Traceback (most recent call last):
  File "/home/ats/Project/mini-AlphaStar/alphastarmini/core/sl/transform_replay_data.py", line 493, in test
    feature, label = getFeatureAndLabel_numpy(obs, func_call)
  File "/home/ats/Project/mini-AlphaStar/alphastarmini/core/sl/transform_replay_data.py", line 135, in getFeatureAndLabel_numpy
    s = Agent.get_state_and_action_from_pickle_numpy(obs)
  File "/home/ats/Project/mini-AlphaStar/alphastarmini/core/arch/agent.py", line 95, in get_state_and_action_from_pickle_numpy
    batch_entities_tensor = Agent.preprocess_state_entity_numpy(obs)
  File "/home/ats/Project/mini-AlphaStar/alphastarmini/core/arch/agent.py", line 355, in preprocess_state_entity_numpy
    entities_array = ArchModel.preprocess_entity_numpy(e_list)
  File "/home/ats/Project/mini-AlphaStar/alphastarmini/core/arch/arch_model.py", line 88, in preprocess_entity_numpy
    return EntityEncoder.preprocess_numpy(e_list)
  File "/home/ats/Project/mini-AlphaStar/alphastarmini/core/arch/entity_encoder.py", line 111, in preprocess_numpy
    return cls.preprocess_in_numpy(entity_list)
  File "/home/ats/Project/mini-AlphaStar/alphastarmini/core/arch/entity_encoder.py", line 524, in preprocess_in_numpy
    order_queue_length_encoding = L.np_one_hot(np.array([order_queue_length]), cls.max_order_queue_length).reshape(1, -1)
  File "/home/ats/Project/mini-AlphaStar/alphastarmini/lib/utils.py", line 280, in np_one_hot
    res = np.eye(nb_classes)[np.array(targets).reshape(-1)]
IndexError: index 9 is out of bounds for axis 0 with size 9```

Firstly I tried Starcraft 4.1.2, but there is a pysc2 websocket timeout issue. Then I tried 4.6. The websocket timeout issue is gone but this index out of bound error arises. 

My OS is ubuntu 20.04.3.

Trained model replays.

Once, I have trained the model, how do I run a game against it say by either a bot or human opponent. I'm unable to figure out how to run the custom agent. Any ideas on how this can be done?

Reproducing the recent SL model featured in README.md

@liuruoze, let me first thank you for your effort to develop a practical version of AlphaStar!

I am really interested in reproducing your recent SL-based agent, which you featured in README.md via several video clips. I’d like to know which StarCraft II version (e.g., 3.16.1) and training values (batch size and the number of epochs) you used when building the SL model.

Thank you again for your amazing work - hopefully, I can contribute to your work, too!

What version of replays?

Hello, please tell me which version of the replays should be loaded into the data folder?

Unable to process action with ability_id: xxxx

您好，我在运行这段代码时

出现了如下的问题

我使用的sc2版本为3.16.1，replay文件是您在Useful-Big-Resources中提供的，系统为ubuntu20.04。请问这个问题该怎么解决呢？

SL Training Results?

Hello,

Great job on this repo! I was wondering if you have any results of the model against bots? Such as winrate for example or sample replays.

Thanks!

如何得到训练结果的GIF

您好，我想知道在训练结束之后如何得到您文档中展示的4个训练结果的GIF
是将训练之后的模型接入到实际游戏中么

Is it possible to share some experiment details about model training ?

Thanks a lot for sharing this awesome project !

I really want to have a try for the training part, but didn't find any experiment details in README or your arxiv paper. Is it possible for you to kindly share some info about the questions below ?

How many disk/memory space should I prepare for the training part?
How long it takes to do supervised/rl train with single or multi gpus?
What level can the trained agent achieve? Can the trained agent beat easy/median/hard bot in sc2 ?

Reproducing results on latest Starcraft version (5.0.9)

Hi, thank you for your work!
I downloaded pretrained weights ("rl_22-02-07_16-26-48.pth") and then i ran:

from alphastarmini.core.rl import rl_vs_inner_bot_mp
rl_vs_inner_bot_mp.test(on_server=P.on_server, replay_path=P.replay_path)

Unfortunately, agent is only building probes, and definitely not acting according to anyhow optimal policy.
I am using latest SC2 version, is this the reason? Is it possible to provide model weights for a new version?

种族的选取

在test_alphastar_replay.py文件中，设置使用‘Terran’，
run_alphastar_replay(on_server=on_server, race_name='Terran')
但“An Introduction of mini-AlphaStar”论文中说的是只能使用'Protoss'。这里是不是应该按照论文中所说：设置种族为'Protoss'
run_alphastar_replay(on_server=on_server, race_name='Protoss')
非常感谢！！

Starcraft SC2Replay data reading problem

I downloaded the replay data according to the tutorial and extracted it into ".SC2Replay" format. However, the run.py file is not read properly and gives the following error

Traceback (most recent call last):
File "/home/czy/Documents/network/reforcement/mini-AlphaStar-main/alphastarmini/core/sl/dataset_pickle.py", line 140, in get_trainable_data
traj_dict = pickle.load(handle)
_pickle.UnpicklingError: invalid load key, '\x1b'.

Does the downloaded data require additional pre-processing?

想请教一下关于预训练的数据问题

大佬您好，想请教下关于与训练的数据问题：
1.通过对回放视频进行处理，可以得到预训练数据，关于这里我想请教问下：这个提取数据，以及出来的特征和标签是暴雪那边给好的吗？也就是这一部分数据打标是怎么来的呢？特征我理解就是当前局面的一些统计特征、地图特征等等，但标签是什么呢？是动作的one-hot形式？
2.如果仅仅是动作的话，怎么实现对整个网络的训练的呢？我理解按照：动作类型——delay——排序队伍——选择动作执行单位——目标单位——目标地址（这里可以是目标单位，或者某一个地方），那么后面几步的标签怎么来呢？还是我理解的哪里不到位，希望能够给出一点指点，感谢。
3.那么后续强化学习的奖励，是否就和这个地方标签如何打标是具有一定的关系了？因为我看论文里面，奖励主要是设置伪奖励，伪奖励除开较为好设计的击杀等，主要是建造顺序这些设置，那么这种设置是否就要和前面的标签结合

mini-AlphaStar v1.04版本编译

您好，我又来了，前一段时间一直在做别的事情，最近打算赶紧研究下mini-AlphaStar。让人惊喜的是mini-AlphaStar已经更新到1.04了！然后我按照readme操作了一番，遇到了如下的问题：

（1）我按照知乎里说的，下载了与我的StarCraftII相同版本的回放文件，然后将回放文件复制到 ./data/replays/，然后运行
transform_replay_data.test，结果在 ./data/replay_data_tensor_new文件夹内没有生成任何文件，并输出如下信息

8bf226bac73e87f9996a71.SC2Replay
replay_path: /home/zhq/Doctor/RL_Project/mini-AlphaStar/data/replays/ffe17a49ecefd2396f634545d63751a0392c3300b81b647dbf4b7285da6925e1.SC2Replay
replay_path: /home/zhq/Doctor/RL_Project/mini-AlphaStar/data/replays/ffe4d74a07f04da3f137d6fd967d25ee31f08f1623d6ae20f663809997660b92.SC2Replay
replay_path: /home/zhq/Doctor/RL_Project/mini-AlphaStar/data/replays/ffea13e5c795065700f7aee8f7427c0fb417e09acdfffe09f9c9b2aab5570f6d.SC2Replay
replay_path: /home/zhq/Doctor/RL_Project/mini-AlphaStar/data/replays/fff40a57d7de2b96da6cb7eb42064c2daf694cbc26c848335ae641f099bac6fa.SC2Replay
100%|███████████████████████████████████████████████████████████████| 200687/200687 [00:02<00:00, 95753.21it/s]
unable to parse websocket frame.
RequestQuit command received.
Closing Application...
end
replay_length_list: []
noop_length_list: []
run over

(2) 在步骤（1）中文件转换没有成功，然后我对 ./data/replays/ 中的文件进行过滤，只保留人族v人族的回放文件，并将过滤后的回放文件保存在 ./data/filtered_replays_1/ 中。然后运行 transform_replay_data.test，结果 replay_data_tensor_new/文件夹中只生成了一个.pt文件。然后假设到目前为之没有错误，继续运行 sl_train_by_tensor.test，但是出了错误，输出如下:

(base) zhq@Ubuntu20:~/Doctor/RL_Project/mini-AlphaStar$ python run.py 
pygame 2.0.1 (SDL 2.0.14, Python 3.8.5)
Hello from the pygame community. https://www.pygame.org/contribute.html
run init
cudnn available
cudnn version 7605
==> Making model..
The number of parameters of model is 2638715
==> Preparing data..
  0%|                                                                                    | 0/1 [00:00<?, ?it/s]replay_path: ./data/replay_data_tensor_new/d55bf6b94d417716ca9caf36f065727efe93aa0059893e1000e9a5afb79a98a2.pt
100%|████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.54it/s]
  0%|                                                                                    | 0/1 [00:00<?, ?it/s]replay_path: ./data/replay_data_tensor_new/d55bf6b94d417716ca9caf36f065727efe93aa0059893e1000e9a5afb79a98a2.pt
100%|█████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 47127.01it/s]
Traceback (most recent call last):
  File "run.py", line 47, in <module>
    sl_train_by_tensor.test(on_server=P.on_server)
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/sl_train_by_tensor.py", line 463, in test
    main_worker(DEVICE)
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/sl_train_by_tensor.py", line 161, in main_worker
    val_set = ConcatDataset(val_list)
  File "/home/zhq/anaconda3/lib/python3.8/site-packages/torch/utils/data/dataset.py", line 200, in __init__
    assert len(datasets) > 0, 'datasets should not be an empty iterable'  # type: ignore
AssertionError: datasets should not be an empty iterable

(3) 直接运行rl_vs_computer_wo_replay.test，则提示如下错误信息

(base) zhq@Ubuntu20:~/Doctor/RL_Project/mini-AlphaStar$ python run.py 
pygame 2.0.1 (SDL 2.0.14, Python 3.8.5)
Hello from the pygame community. https://www.pygame.org/contribute.html
run init
cudnn available
cudnn version 7605
initialed player
initialed teacher
learner trajectories size: 0
learner trajectories size: 0
start_time before training: 2021-12-07 10:26:38
map name: AbyssalReef
player.name: MainPlayer
player.race: Race.protoss
Version: B60604 (SC2.4.01)
Build: May  1 2018 19:24:12
Command Line: '"/home/zhq/StarCraftII/Versions/Base60321/SC2_x64" -listen 127.0.0.1 -port 18844 -dataDir /home/zhq/StarCraftII/ -tempDir /tmp/sc-k2b9f5n_/ -dataVersion 33D9FE28909573253B7FC352CE7AEA40'
Starting up...
Startup Phase 1 complete
learner trajectories size: 0
Startup Phase 2 complete
Creating stub renderer...
Listening on: 127.0.0.1:18844
Startup Phase 3 complete. Ready for commands.
learner trajectories size: 0
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface enabled
Configure: score interface enabled
Configure: render interface disabled
Entering load game phase.
Launching next game.
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
learner trajectories size: 0
learner trajectories size: 0
learner trajectories size: 0
learner trajectories size: 0
learner trajectories size: 0
learner trajectories size: 0
Game has started.
Sending ResponseJoinGame
start_time before reset: 2021-12-07 10:26:46
total_episodes: 1
start_episode_time before is_final: 2021-12-07 10:26:46
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [672,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [673,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [674,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [675,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [676,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [677,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [678,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [679,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [680,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [681,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [682,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [683,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [684,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [685,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [686,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [687,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [688,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [689,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [690,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [691,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [692,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [693,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [694,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [695,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [696,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [697,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [698,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [699,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [700,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [701,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [702,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [703,0,0] Assertion `val >= zero` failed.
unable to parse websocket frame.
RequestQuit command received.
Closing Application...
ActorLoop.run() Exception cause return, Detials of the Exception: CUDA error: device-side assert triggered
Traceback (most recent call last):
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/rl/rl_vs_computer_wo_replay.py", line 175, in run
    player_step = self.player.agent.step_logits(home_obs, player_memory)
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/rl/alphastar_agent.py", line 197, in step_logits
    action, action_logits, new_state, select_units_num = self.step_nn(observation=obs, last_state=last_state)
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/rl/alphastar_agent.py", line 164, in step_nn
    action_logits, action, hidden_state, select_units_num = self.agent_nn.action_logits_by_state(state, single_inference=True,
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/arch/agent.py", line 278, in action_logits_by_state
    action_logits, actions, new_state, select_units_num = self.model.forward(state, batch_size = batch_size,
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/arch/arch_model.py", line 128, in forward
    target_location_logits, target_location = self.location_head(autoregressive_embedding, action_type, map_skip)
  File "/home/zhq/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/arch/location_head.py", line 287, in forward
    location_id = torch.multinomial(target_location_probs, num_samples=1, replacement=True)
RuntimeError: CUDA error: device-side assert triggered

Actor stops!
Learner also stops!
run over

(4) 运行 rl_train_with_replay.test进行 rl训练，则提示如下错误


(base) zhq@Ubuntu20:~/Doctor/RL_Project/mini-AlphaStar$ python run.py 
pygame 2.0.1 (SDL 2.0.14, Python 3.8.5)
Hello from the pygame community. https://www.pygame.org/contribute.html
run init
No models are found!
No models are found!
learner trajectories size: 0
learner trajectories size: 0
start_time before training: 2021-12-07 11:09:13
map name: AbyssalReef
player.name: MainPlayer
opponent.name: MainPlayer
player.race: Race.protoss
opponent.race: Race.protoss
Version: B60604 (SC2.4.01)
Build: May  1 2018 19:24:12
Command Line: '"/home/zhq/StarCraftII/Versions/Base60321/SC2_x64" -listen 127.0.0.1 -port 24993 -dataDir /home/zhq/StarCraftII/ -tempDir /tmp/sc-2o9j_vnz/'
Starting up...
Startup Phase 1 complete
learner trajectories size: 0
Startup Phase 2 complete
Creating stub renderer...
Listening on: 127.0.0.1:24993
Startup Phase 3 complete. Ready for commands.
learner trajectories size: 0
Version: B60604 (SC2.4.01)
Build: May  1 2018 19:24:12
Command Line: '"/home/zhq/StarCraftII/Versions/Base60321/SC2_x64" -listen 127.0.0.1 -port 16658 -dataDir /home/zhq/StarCraftII/ -tempDir /tmp/sc-k86c8jn9/'
Starting up...
Startup Phase 1 complete
learner trajectories size: 0
Startup Phase 2 complete
Creating stub renderer...
Listening on: 127.0.0.1:16658
Startup Phase 3 complete. Ready for commands.
learner trajectories size: 0
Requesting to join a multiplayer game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface enabled
Configure: score interface enabled
Configure: render interface disabled
Requesting to join a multiplayer game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface enabled
Configure: score interface enabled
Configure: render interface disabled
Entering load game phase.
Create game with map path: AbyssalReefLE.SC2Map
Entering load game phase.
learner trajectories size: 0
Attempting to join a game with the map: Ladder2017Season4/AbyssalReefLE.SC2Map
learner trajectories size: 0
learner trajectories size: 0
learner trajectories size: 0
learner trajectories size: 0
learner trajectories size: 0
Game has started.
Sending ResponseJoinGame
Game has started.
Sending ResponseJoinGame
start_time before reset: 2021-12-07 11:09:24
total_episodes: 1
learner trajectories size: 0
Version: B60604 (SC2.4.01)
Build: May  1 2018 19:24:12
Command Line: '"/home/zhq/StarCraftII/Versions/Base60321/SC2_x64" -listen 127.0.0.1 -port 19791 -dataDir /home/zhq/StarCraftII/ -tempDir /tmp/sc-5ebl19qe/ -dataVersion 33D9FE28909573253B7FC352CE7AEA40 -eglpath libEGL.so'
Starting up...
Startup Phase 1 complete
learner trajectories size: 0
Startup Phase 2 complete
Attempting to initialize EGL from file libEGL.so ...
Successfully loaded EGL library!
Successfully initialized display on device idx: 0, EGL version: 1.5

Running CGLSimpleDevice::HALInit...
Calling glGetString: 0x7f7351fc85e0
Version: 4.6.0 NVIDIA 460.91.03
Vendor: NVIDIA Corporation
Renderer: GeForce GTX 1650/PCIe/SSE2
OpenGL initialized!
Disabling compressed textures
Listening on: 127.0.0.1:19791
Startup Phase 3 complete. Ready for commands.
learner trajectories size: 0
replay_path: /home/zhq/Doctor/RL_Project/mini-AlphaStar/data/filtered_replays_1/df4f689685d451cd2ae60f6e19af81dde8b812ddb42264cd1ae468e91796c677.SC2Replay
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface enabled
Configure: score interface enabled
Configure: render interface disabled
Launching next game.
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
learner trajectories size: 0
learner trajectories size: 0
Starting replay 'TempStartReplay.SC2Replay'
learner trajectories size: 0
learner trajectories size: 0
Game has started.
learner trajectories size: 0
start_episode_time before is_final: 2021-12-07 11:09:31
unable to parse websocket frame.
RequestQuit command received.
Closing Application...
unable to parse websocket frame.
RequestQuit command received.
Closing Application...
RequestQuit command received.
Closing Application...
unable to parse websocket frame.
ActorLoop.run() Exception cause return, Detials of the Exception: probability tensor contains either `inf`, `nan` or element < 0
Traceback (most recent call last):
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/rl/actor_plus_z.py", line 242, in run
    player_step = self.player.agent.step_logits(home_obs, player_memory)
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/rl/alphastar_agent.py", line 197, in step_logits
    action, action_logits, new_state, select_units_num = self.step_nn(observation=obs, last_state=last_state)
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/rl/alphastar_agent.py", line 164, in step_nn
    action_logits, action, hidden_state, select_units_num = self.agent_nn.action_logits_by_state(state, single_inference=True,
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/arch/agent.py", line 278, in action_logits_by_state
    action_logits, actions, new_state, select_units_num = self.model.forward(state, batch_size = batch_size,
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/arch/arch_model.py", line 117, in forward
    action_type_logits, action_type, autoregressive_embedding = self.action_type_head(lstm_output, scalar_context)
  File "/home/zhq/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/arch/action_type_head.py", line 157, in forward
    action_type = torch.multinomial(action_type_probs.reshape(batch_size, -1), 1)
RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

Actor stops!
Learner also stops!
run over

运行scalar_encoder.test()结果输出非法值

您好，我通过test.py运行scalar_encoder.test()函数，打印embedded_scalar，结果如下

(base) zhq@Ubuntu20:~/Doctor/RL_Project/mini-AlphaStar$ python test.py 
pygame 2.0.1 (SDL 2.0.14, Python 3.8.5)
Hello from the pygame community. https://www.pygame.org/contribute.html
test init
embedded_scalar: tensor([[nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
         nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
         nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
         nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
         nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
         nan, nan, nan, nan, nan, nan, nan, nan],
        [nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
         nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
         nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
         nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
         nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
         nan, nan, nan, nan, nan, nan, nan, nan]], grad_fn=<ReluBackward0>)
embedded_scalar.shape: torch.Size([2, 128])
test over

我运行您代码库中 rl_vs_computer_wo_replay.test()，运行出错，我在想：是不是scalar_encoder.test()中打印embedded_scalar出现的错误的原因也是导致 rl_vs_computer_wo_replay.test()出错的原因呢？

运行sl_train_by_pickle.py 出错

太感谢了！已经成功对原始Replays进行了过滤，并成功运行transform_replay_data.py产生了5个.pickle文件。还有个问题，就是接下来我注释掉transform_replay_data.py，反注释sl_train_by_pickle.py，然后运行代码。出现了如下错误：

(base) zhq@Ubuntu20:~/Doctor/RL_Project/mini-AlphaStar$ python run.py 
arch init
sl init
rl init
lt init
core init
transformer init
third init
lib init
alphastarmini init
pygame 2.0.1 (SDL 2.0.14, Python 3.8.5)
Hello from the pygame community. https://www.pygame.org/contribute.html
run init
length of replay_files: 5
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 27.47it/s]
training_size: 4
val_size: 1
torch.cuda.device_count(): 1
Traceback (most recent call last):
  File "run.py", line 46, in <module>
    sl_train_by_pickle.test(on_server=P.on_server)
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/sl_train_by_pickle.py", line 224, in test
    train_for_val(replays, None, agent)  # select the best hyper-parameters
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/sl_train_by_pickle.py", line 132, in train_for_val
    traj = next(iter(train_loader)) # 通过iter()函数获取这些可迭代对象的迭代器。然后我们可以对获取到的迭代器不断使??next()函数来获取下??条数据。iter()函数实际上就是调??了可迭代对象的
  File "/home/zhq/anaconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 517, in __next__
    data = self._next_data()
  File "/home/zhq/anaconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 557, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/home/zhq/anaconda3/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/zhq/anaconda3/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/dataset_pickle.py", line 110, in __getitem__
    replay = self._get_random_trajectory(index)
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/dataset_pickle.py", line 105, in _get_random_trajectory
    the_item = next(iter(traj_loader))
  File "/home/zhq/anaconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 517, in __next__
    data = self._next_data()
  File "/home/zhq/anaconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 557, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/home/zhq/anaconda3/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/zhq/anaconda3/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/dataset_pickle.py", line 73, in __getitem__
    feature, label = obs2feature(obs, self.agent)
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/dataset_pickle.py", line 31, in obs2feature
    feature = Feature.state2feature(s)
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/feature.py", line 60, in state2feature
    feature_3 = map_data.reshape(batch_size, AHP.map_channels * AHP.minimap_size * AHP.minimap_size)
RuntimeError: shape '[1, 139264]' is invalid for input of size 73728

Start error

Hello, when I run the command python run.py, I get this error:
ActorLoop.run() Exception cause return, Detials of the Exception: Failed to connect to the SC2 websocket. Is it up? Traceback (most recent call last): File "/Users/Administrator/Documents/less/lesson/alphastar/part_3/mini/mini-AlphaStar/alphastarmini/core/rl/against_computer.py", line 106, in run with self.create_env_one_player(self.player) as env: File "/Users/Administrator/Documents/less/lesson/alphastar/part_3/mini/mini-AlphaStar/alphastarmini/core/rl/against_computer.py", line 268, in create_env_one_player random_seed=random_seed) File "/Applications/anaconda3/envs/torch_1_7/lib/python3.7/site-packages/pysc2/env/sc2_env.py", line 287, in __init__ self._launch_game() File "/Applications/anaconda3/envs/torch_1_7/lib/python3.7/site-packages/pysc2/env/sc2_env.py", line 358, in _launch_game for interface in self._interface_options] File "/Applications/anaconda3/envs/torch_1_7/lib/python3.7/site-packages/pysc2/env/sc2_env.py", line 358, in <listcomp> for interface in self._interface_options] File "/Applications/anaconda3/envs/torch_1_7/lib/python3.7/site-packages/pysc2/run_configs/platforms.py", line 88, in start self, exec_path=exec_path, version=self.version, **kwargs) File "/Applications/anaconda3/envs/torch_1_7/lib/python3.7/site-packages/pysc2/lib/sc_process.py", line 143, in __init__ self._host, self._port, self, timeout_seconds=timeout_seconds) File "/Applications/anaconda3/envs/torch_1_7/lib/python3.7/site-packages/pysc2/lib/remote_controller.py", line 146, in __init__ sock = self._connect(host, port, proc, timeout_seconds) File "/Applications/anaconda3/envs/torch_1_7/lib/python3.7/site-packages/pysc2/lib/stopwatch.py", line 212, in _stopwatch return func(*args, **kwargs) File "/Applications/anaconda3/envs/torch_1_7/lib/python3.7/site-packages/pysc2/lib/remote_controller.py", line 180, in _connect raise ConnectError("Failed to connect to the SC2 websocket. Is it up?") pysc2.lib.remote_controller.ConnectError: Failed to connect to the SC2 websocket. Is it up?.
What could be the problem? Thank you

Unable to Process Replays?

Just discovered this repo, and I'm extremely interested in getting started with SC2 RL, but I'm quickly finding that most of these repo's aren't maintained anymore/they're from versions of the game LONG ago, so I'm hoping to get some insight.

Now, using this repo, and running Windows 10, Python 3.10, and SC2 version 5.0.9, my run.py looks like this:

Pretty standard, I'm just trying to run the transform_replay_data part. I've also updated the path where the replays are read from in the transform_replay_data.py file to flags.DEFINE_string("no_server_replay_path", "C:\\Program Files (x86)\\StarCraft II\\Replays\\", "path of replay data"). (Double slashes re: Windows).

I run the script, and it looks like it's trying to run, but essentially, it just flies through all the replay files I have in that directory, but it doesn't actually "do" anything, it just looks like this at the end:

I don't see any errors or anything to suggest what it might be, but my guess is it has something to do with the version of replays vs the version of my current game. I've seen in previous posts that you need to actually open the replays in order to download the older versions of the games, which I have done already, and I can see the added version in my Versions directory, but still no dice.

The maps I've downloaded include:

....in case that's relevant. I've also read online that there's a difference between local maps files, and battle.net map files as the latter is played online, and uses a .s2ma extension. In order to download the maps for each replay, the best advice I saw was to enter -PrePopulateCache as an additional command argument in the Battle.net > Game Options settings which downloads those cache files for you, so I've done that as well.

Any ideas on what a guy can try to troubleshoot this? Any help would be greatly appreciated as I'd love to explore this space using the current versions of everything! lol Thanks!

运行run.py时遇到的报错

我在网易官网下载了最新的StarCraft 2客户端，并且把下载的replay放到了mini-AlphaStar-main\data\filtered_replays_1文件夹下，运行run.py中的第一步时出现了如下报错，请问要如何解决？

Target_location_loss is always the same

Hello. I wonder if you could help me. I parsed 100 ladder replays of matches for both sides (protoss v protoss only), in total 200 parsed files, version is 4.10.0. When training for over 6 epochs (600 iterations) all losses change (go down usually), but target_location_loss always stays at 11.090355 no matter what i do. And then if i try to launch RL training using replays (on total loss ~24) model just moves camera, no other commands issued. I'm just curious is it a bug, regarding versions of replays, or i'm doing something else wrong

如何得到训练rl模型对内置ai的胜率

我希望能得到模型对ai的胜率，因此我将
finally:
print("Parameter_Server end:")
print("--------------------")
print("mean_points_list:", mean_points_list)
print("latest_mean_points:", latest_mean_points)
print("max_mean_points:", max_mean_points)
print("win_rate_list:", win_rate_list)
print("latest_win_rate:", latest_win_rate)
print("max_win_rate:", max_win_rate)
print("--------------------")
这一段代码反注释并重新训练rl模型，但是最后输出的结果为0，我想知道怎么做才能得到您文章中的胜率结果

回放文件Replays

   您好，关于“data/Replays/filtered_replays_1”中的回放文件是怎么得到的呀？ 我尝试的操作是：利用sc2client-proto/samples/replay-api/download_replays.py下载与我的StarCraftII（4.9.3）对应版本的回放文件，但是运行download_replays.py文件时提示“Failed to get oauth access token.”，其中我的运行命令是 python download_replays.py --key=我的battle.net邮箱账号 --secret=我的battle.net账号密码 --version=4.9.3 --replays_dir=/home/zhq/Doctor/RL_Project/s2client-proto/samples/replay-api/replays --extract。
   请问下，这种情况是怎么回事呀？

CUDA out of memory

Hi, im running on windows 10 and using latest starcraft 2. The error shown as below when I run "python run.py"

(torch_1_5) PS C:\Users\alexa\Downloads\mini-AlphaStar-main> python run.py
pygame 2.1.2 (SDL 2.0.18, Python 3.7.11)
Hello from the pygame community. https://www.pygame.org/contribute.html
run init
cudnn available
cudnn version 7604
initialed player
initialed teacher
start_time before training: 2022-01-01 18:11:32
map name: Simple64
player.name: MainPlayer
player.race: Race.protoss
start_time before reset: 2022-01-01 18:13:12
total_episodes: 1
start_episode_time before is_final: 2022-01-01 18:13:13
ActorLoop.run() Exception cause return, Detials of the Exception: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 2.00 GiB total capacity; 1.27 GiB already allocated; 0 bytes free; 1.33 GiB reserved in total by PyTorch)
Traceback (most recent call last):
File "C:\Users\alexa\Downloads\mini-AlphaStar-main\alphastarmini\core\rl\rl_vs_computer_wo_replay.py", line 213, in run
player_step = self.player.agent.step_from_state(state, player_memory)
File "C:\Users\alexa\Downloads\mini-AlphaStar-main\alphastarmini\core\rl\alphastar_agent.py", line 235, in step_from_state
hidden_state=hidden_state)
File "C:\Users\alexa\Downloads\mini-AlphaStar-main\alphastarmini\core\arch\agent.py", line 299, in action_logits_by_state
return_logits = True)
File "C:\Users\alexa\Downloads\mini-AlphaStar-main\alphastarmini\core\arch\arch_model.py", line 134, in forward
entity_embeddings, embedded_entity, entity_nums = self.entity_encoder(state.entity_state)
File "C:\Users\alexa\anaconda3\envs\torch_1_5\lib\site-packages\torch\nn\modules\module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "C:\Users\alexa\Downloads\mini-AlphaStar-main\alphastarmini\core\arch\entity_encoder.py", line 390, in forward
unit_types_one = torch.nonzero(batch, as_tuple=True)[-1]
RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 2.00 GiB total capacity; 1.27 GiB already allocated; 0 bytes free; 1.33 GiB reserved in total by PyTorch)

run over

Can i know what is the problem here and what is the solution? Thanks

Issue with result reproduction

I have run the algorithm for around more than 30 days on an NVIDIA DGX server, with the configuration of 500GB RAM and 8 Tesla GPUs with 32 GB VRAM each. The algorithm still seems to be losing all the matches against the level 2 difficulty bot.
The source code has been used from the mini-AlphaStar repo, version 1.07, on SC2 game version 4.10.0.
Kindly advise me on how to proceed with respect to the same.
Link to the output logs for the latest batch of RL training against the level 2 bot - https://learnermanipal-my.sharepoint.com/:u:/g/personal/rahat_santosh1_learner_manipal_edu/EVi0z-QKI19Ogh8LK8PnbloB6xEdedXNc7_a-AMuTDejOQ?e=bwQRzr

Some questions in Rl-training

1.问一下你们是在什么系统上运行的呢？我在使用gpu运行rl_vs_inner_bot时会报错，查了下似乎是在主进程中把模型参数放到了GPU上会报错，Windows似乎不支持在CUDA上进行multiprocess的操作？有什么好的解决方法吗？
RuntimeError:cuda runtime error(801) : operation not supported at ...
2.请问可以提供RL训练过程的曲线、结果以及相关技巧吗？我在arxiv和github上似乎没有找到rl训练过程的更多内容。直接运行开源程序rl_vs_inner_bot模块，Tensorboard的结果似乎存在问题？

What is the meaning of Features and Labels?

Hello, My name is Chan Uk Lee. I am a Korea Maritime and Ocean University Master course student.

I am very interested in your artificial intelligence Mini-Alphastar.

but it is very difficult for me to understand.

So I have a question.

Transform replay data python file get Features and Label.

What is the meaning of Features and Labels? and How can i get Features and Labels?

Last, How are Features and Labels used in supervised learning?

Starcraft 2 version problem

Hi, I am having latest starcraft 2 version which is 5.0.8 on windows 10, when I try to run " python run.py" with codes below:

# 5. we use SL model and replays to do reinforcement learning

from alphastarmini.core.rl import rl_train_with_replay
rl_train_with_replay.test(on_server=P.on_server, replay_path=P.replay_path)

Result with error below:

ActorLoop.run() Exception cause return, Detials of the Exception: Unknown game version: 3.16.1. Known versions: ['latest'].
Traceback (most recent call last):
File "C:\Users\alexa\Downloads\mini-AlphaStar-main\alphastarmini\core\rl\actor_plus_z.py", line 130, in run
run_config = run_configs.get(version=self.replay_version) # the replays released by blizzard are all 3.16.1 version
File "C:\Users\alexa\anaconda3\envs\torch_1_5\lib\site-packages\pysc2\run_configs_init_.py", line 40, in get
return max(configs.values(), key=lambda c: c.priority())(version=version)
File "C:\Users\alexa\anaconda3\envs\torch_1_5\lib\site-packages\pysc2\run_configs\platforms.py", line 121, in init
version=version, cwd="Support64")
File "C:\Users\alexa\anaconda3\envs\torch_1_5\lib\site-packages\pysc2\run_configs\platforms.py", line 63, in init
data_dir=base_dir, tmp_dir=None, version=version, cwd=cwd, env=env)
File "C:\Users\alexa\anaconda3\envs\torch_1_5\lib\site-packages\pysc2\run_configs\lib.py", line 112, in init
self.version = self._get_version(version)
File "C:\Users\alexa\anaconda3\envs\torch_1_5\lib\site-packages\pysc2\run_configs\lib.py", line 220, in _get_version
versions = self.get_versions(containing=game_version)
File "C:\Users\alexa\anaconda3\envs\torch_1_5\lib\site-packages\pysc2\run_configs\platforms.py", line 109, in get_versions
containing, sorted(ret.keys())))
ValueError: Unknown game version: 3.16.1. Known versions: ['latest'].

run over

Which I know it needs SC2 3.16.1 version to work, but I cannot get older version of Starcraft 2. Please advice me on this. Thanks!!

代码的运行

运行run.py后出错，终端提示如下：
File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/sl/test_alphastar_replay.py", line 192, in run_alphastar_replay
replay_files = os.listdir(REPLAY_PATH)
FileNotFoundError: [Errno 2] No such file or directory: './data/Replays/replays_paper_ready/Final/Protoss/'

我已经把StarCraftII/Replays文件夹放入了mini-AlphaStar/data文件夹中，但依然找不到相应的路径，请问下“Replays/replays_paper_ready/Final/Protoss/” 回放的数据我能怎么获得呢？

运行rl_vs_inner_bot_mp.py没有输出

作者您好，我已经用sl模型训练得到了rl模型，现在我想看一下这个模型和bot的对战效果，但当我试图运行rl_vs_inner_bot_mp.py时，程序没有任何输出就退出了。图片中的运行界面是我在win11系统用pycharm运行得到的，我在实验室的服务器上进行训练后，将训练得到的model中的rl模型下载到我的电脑中并放在model文件夹内，运行rl_vs_inner_bot_mp.py，产生了这样的问题。想问下您这个问题怎么解决，感谢！

Missing relu activation functions.

mini-AlphaStar/alphastarmini/core/arch/spatial_encoder.py

Lines 73 to 75 in 04f755b

    
           x = self.ds_1(x) 
        
           x = self.ds_2(x) 
        
           x = self.ds_3(x)

Hi Liuruoze,
Thank you for the amazing work. I found your code is kind different from what stated in detailed-architecture.txt. Is there a reason to omit the relu between convolution layers? Also I have checked the very similar implemtation from Tencent. https://github.com/dachidahu/SC2AgentsZoo2/blob/1300af878cab00d16b0f266753b10acdbfee0bbc/agent_TLeagueFormal14/TPolicies/tpolicies/net_zoo/mnet_v6/utils.py#L408-L412.
As you can see the relu is applied for each of convolution layer. (relu is default activation function in tensorflow 1.15 https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/contrib/layers/conv2d)

Thanks.

mini-AlphaStar V1.05编译报错

这次我是用 3.16.1版本的replay files，v4.1.2版本的StarCraftII，然后直接运行 rl_train_with_replay.test，提示如下错误：

(base) zhq@Ubuntu20:~/Doctor/RL_Project/mini-AlphaStar$ python run.py 
pygame 2.0.1 (SDL 2.0.14, Python 3.8.5)
Hello from the pygame community. https://www.pygame.org/contribute.html
run init
cudnn available
cudnn version 7605
learner trajectories size: 0
learner trajectories size: 0
start_time before training: 2021-12-07 17:23:35
map name: AbyssalReef
player.name: MainPlayer
opponent.name: Historical
player.race: Race.protoss
opponent.race: Race.protoss
Version: B60604 (SC2.4.01)
Build: May  1 2018 19:24:12
Command Line: '"/home/zhq/StarCraftII/Versions/Base60321/SC2_x64" -listen 127.0.0.1 -port 21930 -dataDir /home/zhq/StarCraftII/ -tempDir /tmp/sc-7nrvdwyr/ -dataVersion 33D9FE28909573253B7FC352CE7AEA40'
Starting up...
Startup Phase 1 complete
learner trajectories size: 0
Startup Phase 2 complete
Creating stub renderer...
Listening on: 127.0.0.1:21930
Startup Phase 3 complete. Ready for commands.
learner trajectories size: 0
Version: B60604 (SC2.4.01)
Build: May  1 2018 19:24:12
Command Line: '"/home/zhq/StarCraftII/Versions/Base60321/SC2_x64" -listen 127.0.0.1 -port 18821 -dataDir /home/zhq/StarCraftII/ -tempDir /tmp/sc-4olvry4e/ -dataVersion 33D9FE28909573253B7FC352CE7AEA40'
Starting up...
Startup Phase 1 complete
learner trajectories size: 0
Startup Phase 2 complete
Creating stub renderer...
Listening on: 127.0.0.1:18821
Startup Phase 3 complete. Ready for commands.
learner trajectories size: 0
Requesting to join a multiplayer game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface enabled
Configure: score interface enabled
Configure: render interface disabled
Requesting to join a multiplayer game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface enabled
Configure: score interface enabled
Configure: render interface disabled
Entering load game phase.
Entering load game phase.
Create game with map path: AbyssalReefLE.SC2Map
learner trajectories size: 0
Attempting to join a game with the map: Ladder2017Season4/AbyssalReefLE.SC2Map
learner trajectories size: 0
learner trajectories size: 0
learner trajectories size: 0
learner trajectories size: 0
learner trajectories size: 0
Game has started.
Sending ResponseJoinGame
Game has started.
Sending ResponseJoinGame
start_time before reset: 2021-12-07 17:23:46
total_episodes: 1
learner trajectories size: 0
Version: B55958 (SC2.3.16)
Build: Jul 31 2017 13:19:41
Command Line: '"/home/zhq/StarCraftII/Versions/Base55958/SC2_x64" -listen 127.0.0.1 -port 17945 -dataDir /home/zhq/StarCraftII/ -tempDir /tmp/sc-s6etrbvj/ -dataVersion 5BD7C31B44525DAB46E64C4602A81DC2 -eglpath libEGL.so'
Starting up...
Startup Phase 1 complete
learner trajectories size: 0
Startup Phase 2 complete
Creating stub renderer...
Listening on: 127.0.0.1:17945 (17945)
Startup Phase 3 complete. Ready for commands.
learner trajectories size: 0
replay_path:------------- /home/zhq/Doctor/RL_Project/mini-AlphaStar/data/Replays/filtered_replays_1/fdcd9f6c099e6efd1d490be9299f3a19451aad9b56ddc48acf52f7233fbe8561.SC2Replay
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface enabled
Configure: score interface enabled
Configure: render interface disabled
Launching next game.
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
learner trajectories size: 0
learner trajectories size: 0
Starting replay 'TempStartReplay.SC2Replay'
learner trajectories size: 0
learner trajectories size: 0
Game has started.
start_episode_time before is_final: 2021-12-07 17:23:52
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [928,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [929,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [930,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [931,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [932,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [933,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [934,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [935,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [936,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [937,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [938,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [939,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [940,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [941,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [942,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [943,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [944,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [945,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [946,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [947,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [948,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [949,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [950,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [951,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [952,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [953,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [954,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [955,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [956,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [957,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [958,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [959,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [384,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [385,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [386,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [387,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [388,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [389,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [390,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [391,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [392,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [393,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [394,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [395,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [396,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [397,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [398,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [399,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [400,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [401,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [402,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [403,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [404,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [405,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [406,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [407,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [408,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [409,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [410,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [411,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [412,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [413,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [414,0,0] Assertion `val >= zero` failed.
/opt/conda/conda-bld/pytorch_1616554788289/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:197: sampleMultinomialOnce: block: [0,0,0], thread: [415,0,0] Assertion `val >= zero` failed.
unable to parse websocket frame.
RequestQuit command received.
Closing Application...
unable to parse websocket frame.
RequestQuit command received.
Closing Application...
RequestQuit command received.
unable to parse websocket frame.
Closing Application...
ActorLoop.run() Exception cause return, Detials of the Exception: CUDA error: device-side assert triggered
Traceback (most recent call last):
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/rl/actor_plus_z.py", line 282, in run
    player_step = self.player.agent.step_from_state(state, player_memory)
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/rl/alphastar_agent.py", line 234, in step_from_state
    action_logits, action, hidden_state, select_units_num = self.agent_nn.action_logits_by_state(state,
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/arch/agent.py", line 328, in action_logits_by_state
    action_logits, actions, new_state, select_units_num = self.model.forward(state, batch_size = batch_size,
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/arch/arch_model.py", line 128, in forward
    target_location_logits, target_location = self.location_head(autoregressive_embedding, action_type, map_skip)
  File "/home/zhq/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/zhq/Doctor/RL_Project/mini-AlphaStar/alphastarmini/core/arch/location_head.py", line 293, in forward
    location_id = torch.multinomial(target_location_probs, num_samples=1, replacement=True)
RuntimeError: CUDA error: device-side assert triggered

Actor stops!
Learner also stops!
run over

请问这是这是为什么呀

liuruoze / mini-alphastar Goto Github PK

mini-alphastar's Introduction

mini-alphastar's People

Contributors

Stargazers

Watchers

Forkers

mini-alphastar's Issues

# 6. we use SL model and replays to do reinforcement learning

# 5. we use SL model and replays to do reinforcement learning

Recommend Projects

Recommend Topics

Recommend Org