mihahauke / vdaic2017

:godmode: Helpful files for Visual Doom AI Competition 2017 :godmode:

Sample and Historical submissions for Visual Doom AI Competition 2017 at CIG2017

Submissions require Docker. All images except for the host require quite recent Nvidia drivers and nvidia-docker to run CUDA.

GUI forwarding from Docker was tested only on a Linux host and it's not guaranteed to work properly on other systems at the moment.

Quick Start

./build.sh host
./build.sh random
./run.sh host -w
./run.sh random  # in a different terminal

Building and Running the Images

We have prepared two wrapper scripts which will build and run docker images with agents and the host.

To run gui on X11 properly make sure that GID and UID are the same in Dockerfile as in your host.

To build and launch given container run:

DIR=host # or any other directory with Dockerfile (e.g., random, f1, no_host, or intelact)

# Build a docker image named cig2017_${DIR}
./build.sh ${DIR} 

# Run a docker image named cig2017_${DIR}
./run.sh ${DIR}

Provided Images

By default, the agents connect to localhost and have the GUI window disabled. To customize this behavior change _vizdoom.cfg file and rebuild the image.

• host - the image will be used for the initialization of the game. All agents are supposed to connect to the host. By default, the host creates a 10-minutes deathmatch for 1 player on map01 with no bots. To change this behaviour use run.sh's optional parameters:

# Sample usage (a 12-minutes deathmatch for 8 players on map 1):
./run.sh host -p 8 -t 12 -m 1

usage: Host script for ViZDoom Copmetition at CIG 2017. [-h] [-b BOTS_NUM]
                                                        [-p PLAYERS_NUM]
                                                        [-m MAP]
                                                        [-t TIMELIMIT] [-c]
                                                        [-w]

optional arguments:
  -h, --help            show this help message and exit
  -b BOTS_NUM, --bots BOTS_NUM
                        number of bots to add [0,15] (default: 0)
  -p PLAYERS_NUM, --players PLAYERS_NUM
                        number of players [1,16] (default: 1)
  -m MAP, --map MAP     map number [1,5] (default: 1)
  -t TIMELIMIT, --time TIMELIMIT
                        timelimit in minutes [1,999] (default: 10)
  -c, --console         enable console (default: False)
  -w, --watch           roam the map as a ghost spectator (default: False)

• random - a random agent which connects to the host and does not do anything smart. By changing the mode to ASYNC_SPECTATOR and enabling window visibility in _vizdoom.cfg you can replace the random.
• no_host - random agent which does NOT connect to the host - it hosts a game for itself and can add built-in bots. This image will not be used by us but may be useful for training the agents. It runs faster since it is synchronized (mode=PLAYER)
• f1 - the winner entry of the ViZDoom Competition 2016 Limitted Deathmatch by Yuxin Wu and Yuandong Tian,
• intelact - the winner entry of the ViZDoom Competition 2016 Full Deathmatch (track 2) by Alexey Dosovitskiy and Vladlen Koltun.

External files

Some entries were too big to be shared on github, so we provide them externally:

• clyde - the 3rd place of the ViZDoom Competition 2016 Limitted Deathmatch by Dino Ratcliffe

• 2017 submissions - best submissions of Visual Doom AI Competitions 2017 (both tracks)

vdaic2017's Issues

Add support for multiple wads.

cannot watch?

Hi,

I just followed all the instructions, but there is error when running sudo ./run.sh host -w:

Starting vizdoom CIG 2017 host for 1 player.
   WAD:        cig2017.wad
   TIMELIMIT:  10 min.
   MAP:        map01
   PLAYERS:    1
   BOTS:       0
   CONSOLE:    False

No protocol specified
No protocol specified
xcb_connection_has_error() returned true
AL lib: (WW) alc_initconfig: Failed to initialize backend "pulse"
Press 'Q' to abort network game synchronization.
Got connect from node 1.                
Waiting for players: .. [ 2/ 2]
Sending all here.                       

Done waiting.
Go                                      
Total players: 2                        

Found SampleRandomAgent (node 1, player 2)


No protocol specified
Could not initialize SDL video:
Failed to connect to the Mir Server

Traceback (most recent call last):
  File "./host.py", line 110, in <module>
    game.init()
vizdoom.vizdoom.ViZDoomErrorException: Could not initialize SDL video:
Failed to connect to the Mir Server

I've correctly installed libsdl2-dev and the "native" vizdoom runs well with visible window. What can be the reason for the failure?

And the host works fine fine without the "-w" option.

libGL error: No matching fbConfigs or visuals found

Ran into this issue when running ./run.sh host -w:
...
player 1 of 2 (2 nodes)
Use changemap instead. Map is for single-player only.
VIZ_Init: instance id: foDC642ZOg, async: 1, input: 1
VIZ_MQInit: Init message queues.
VIZ_SMInit: Init shared memory.
VIZ_ScreenInit: width: 1024, height: 576, pitch: 1024, format: CRCGCB
libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast

It stalls at the last line with no visual. The line that's been executed to run the docker image is:
nvidia-docker run --net=host -ti --rm --name cig2017_host --env=DISPLAY --privileged cig2017_host -w

Local system environment:
Ubuntu 16.04LTS, Nvidia Graphics Driver 375.26

Docker: apt-get update cache is not kept between installs

Ran into an issue while trying to run (and create) a few dockerfiles.

There are many instances where an apt-get update and an apt-get install ... are run on separate lines. This causes sync issues. For example, after the initial apt-get install {system_updates} there was a subsequent apt-get install...

Step 5/20 : RUN apt-get install -y python-dev python python-pip
 ---> Running in e214c2e0e6cd
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package python-dev
E: Unable to locate package python
E: Unable to locate package python-pip

Did some google'ing and found that docker prefers update and install commands should always be chained with an &&. Therefore, many of the Dockerfile commands should be

RUN apt-get update && apt-get install -y python-dev python python-pip

vs

RUN apt-get update
RUN apt-get install -y python-dev python python-pip

libGL error

I have followed the steps to build the host image, random image, and no_host image. When I set window_visible : false, everything is OK but display the the warning log:

./run.sh no_host
 Gtk-Message: Failed to load module "canberra-gtk-module"
AL lib: (WW) alc_initconfig: Failed to initialize backend "pulse"

Meanwhile, when I set window_visible: true, something fails:

./run.sh no_host 
Gtk-Message: Failed to load module "canberra-gtk-module"
AL lib: (WW) alc_initconfig: Failed to initialize backend "pulse"
libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast

what's wrong with the GUI in vizdoom, and how can I fix this problem?

f1 and clyde agents don't seem to be working

edit: issue with clyde was fixed with a full redo of the docker image. Out-of-sync issue was fixed by using wad from this repo not the vizdoom repo..

What I expected to happen

• F1 forzen/stuck at spawn.

This is what it looks like when view_window is true. Screen stays like this.

After about a minute, that screen changes. F1 respawns. Then F1 continuously walks forward.

What happened

• F1 performed poorly, seemingly randomly

Steps to reproduce

• Clone repository
• Install nvidia-docker and docker as proscribed
• Install clyde from provided dropbox link
• sudo ./build.sh { clyde | host | f1 } as directed with no errors
• Run 3 terminal windows. {sudo ./run.sh clyde | sudo ./run.sh f1 | sudo ./run.sh -c -p 2 -b 6 -li 1 -r .}

Output of each session

host

host.py:119: UserWarning: Recording won't work properly with bots!
  warn("Recording won't work properly with bots!")
Starting vizdoom CIG 2017 host for 2 players.
Configuration:
╒════════════════════╤═════════════╕
│ WAD                │ cig2017.wad │
├────────────────────┼─────────────┤
│ TIMELIMIT (min)    │ 10          │
├────────────────────┼─────────────┤
│ MAP                │ map01       │
├────────────────────┼─────────────┤
│ PLAYERS            │ 2           │
├────────────────────┼─────────────┤
│ BOTS               │ 6           │
├────────────────────┼─────────────┤
│ CONSOLE            │ True        │
├────────────────────┼─────────────┤
│ RECORDFILE         │ .           │
├────────────────────┼─────────────┤
│ LOG_INTERVAL (min) │ 1.0         │
╘════════════════════╧═════════════╛

ViZDoom 1.1.2 (ZDOOM 2.8.1) - SDL version
Compiled on Jun 15 2017

D_DoomInit: Static RNGseed -1119736683 set.
M_LoadDefaults: Load system defaults.
Gameinfo scan took 0 ms
W_Init: Init WADfiles.
 adding /usr/local/lib/python3.5/dist-packages/vizdoom/vizdoom.pk3, 583 lumps
 adding /usr/local/lib/python3.5/dist-packages/vizdoom/doom2.wad, 2956 lumps
 adding cig2017.wad, 35 lumps
viz_controlled is write protected.
viz_instance_id is write protected.
viz_async is write protected.
viz_window_hidden is write protected.
viz_noxserver is write protected.
viz_nosound is write protected.
viz_nocheat is write protected.
viz_debug is write protected.
I_Init: Setting up machine state.
CPU Vendor ID: GenuineIntel
  Name: Intel(R) Core(TM) i5-6600K CPU @ 3.50GHz
  Family 6, Model 94, Stepping 3
  Features: MMX SSE SSE2 SSE3 SSSE3 SSE4.1 SSE4.2
V_Init: allocate screen.
S_Init: Setting up sound.
ST_Init: Init startup screen.
Checking cmd-line parameters...
S_InitData: Load sound definitions.
G_ParseMapInfo: Load map definitions.
Texman.Init: Init texture manager.
ParseTeamInfo: Load team definitions.
LoadActors: Load actor definitions.
R_Init: Init Doom refresh subsystem.
DecalLibrary: Load decals.
M_Init: Init menus.
P_Init: Init Playloop state.
ParseSBarInfo: Loading default status bar definition.
ParseSBarInfo: Loading custom status bar definition.
D_CheckNetGame: Checking network game status.
Console player number: 0
Press 'Q' to abort network game synchronization.
Got connect from node 1.
Got connect from node 2.
Waiting for players: ... [ 3/ 3]
Sending all here.

Done waiting.
Go
Total players: 3
Selected peer to peer networking mode. (auto)

Found CLYDE (node 1, player 2)
Found F1 (node 2, player 3)


player 1 of 3 (3 nodes)
Use changemap instead. Map is for single-player only.
VIZ_Init: instance id: 4xFWznouOA, async: 1, input: 0
VIZ_MQInit: Init message queues.
VIZ_SMInit: Init shared memory.
VIZ_ScreenInit: width: 1024, height: 576, pitch: 1024, format: CRCGCB
Not using any video driver. Check-mate GUI believers!
Resolution: 1024 x 576


map01 - entryway

15 bots read from bots.cfg
Host running.
Removed all bots
Plissken joined the game
Machete joined the game
McClane joined the game
Leone joined the game
Ripley joined the game
Predator joined the game
McClane killed himself.
CLYDE killed himself.
Machete killed himself.
Leone killed himself.
CLYDE killed himself.
Ripley rode Leone's rocket.
Leone killed himself.
CLYDE killed himself.
F1 rode McClane's rocket.
time: 1.00 minutes
╒══════════╤═════════╕
│   Player │   Frags │
╞══════════╪═════════╡
│        2 │      -3 │
├──────────┼─────────┤
│        3 │       0 │
╘══════════╧═════════╛

CLYDE killed himself.
McClane killed himself.
Plissken rode Ripley's rocket.
Ripley killed himself.
CLYDE killed himself.
Plissken killed himself.
Machete rode Ripley's rocket.
Ripley rode McClane's rocket.
Predator rode Leone's rocket.
F1 rode Machete's rocket.
CLYDE killed himself.
CLYDE killed himself.
Predator rode McClane's rocket.
McClane killed himself.
time: 2.00 minutes
╒══════════╤═════════╕
│   Player │   Frags │
╞══════════╪═════════╡
│        2 │      -7 │
├──────────┼─────────┤
│        3 │       0 │
╘══════════╧═════════╛

Machete rode Ripley's rocket.
F1 killed himself.
Ripley rode McClane's rocket.
CLYDE killed himself.
Predator rode Machete's rocket.
CLYDE killed himself.
Ripley killed himself.
Leone rode Predator's rocket.
CLYDE killed himself.
Predator rode McClane's rocket.
time: 3.00 minutes
╒══════════╤═════════╕
│   Player │   Frags │
╞══════════╪═════════╡
│        2 │     -10 │
├──────────┼─────────┤
│        3 │      -1 │
╘══════════╧═════════╛

Ripley killed himself.
Machete killed himself.
F1 rode Leone's rocket.
Predator killed himself.
CLYDE killed himself.
McClane rode Machete's rocket.
Predator killed himself.
CLYDE killed himself.
Predator killed himself.
McClane killed himself.
Ripley rode McClane's rocket.
CLYDE killed himself.
time: 4.00 minutes
╒══════════╤═════════╕
│   Player │   Frags │
╞══════════╪═════════╡
│        2 │     -13 │
├──────────┼─────────┤
│        3 │      -1 │
╘══════════╧═════════╛

Predator killed himself.
Ripley killed himself.
CLYDE killed himself.
Machete rode Ripley's rocket.
McClane killed himself.
Ripley rode Machete's rocket.
Leone rode Machete's rocket.
Machete killed himself.
CLYDE killed himself.
time: 5.00 minutes
╒══════════╤═════════╕
│   Player │   Frags │
╞══════════╪═════════╡
│        2 │     -15 │
├──────────┼─────────┤
│        3 │      -1 │
╘══════════╧═════════╛

McClane killed himself.
CLYDE killed himself.
Machete rode Predator's rocket.
Predator killed himself.
McClane killed himself.
Machete killed himself.
CLYDE killed himself.
Leone rode Predator's rocket.
McClane killed himself.
CLYDE killed himself.
McClane killed himself.
time: 6.00 minutes
╒══════════╤═════════╕
│   Player │   Frags │
╞══════════╪═════════╡
│        2 │     -18 │
├──────────┼─────────┤
│        3 │      -1 │
╘══════════╧═════════╛

CLYDE killed himself.
Leone rode McClane's rocket.
Ripley rode McClane's rocket.
CLYDE killed himself.
CLYDE killed himself.
Plissken rode Leone's rocket.
Leone killed himself.
time: 7.00 minutes
╒══════════╤═════════╕
│   Player │   Frags │
╞══════════╪═════════╡
│        2 │     -21 │
├──────────┼─────────┤
│        3 │      -1 │
╘══════════╧═════════╛

McClane rode Machete's rocket.
Machete killed himself.
McClane killed himself.
Machete killed himself.
Predator rode McClane's rocket.
McClane killed himself.
CLYDE rode Predator's rocket.
Predator killed himself.
time: 8.00 minutes
╒══════════╤═════════╕
│   Player │   Frags │
╞══════════╪═════════╡
│        2 │     -21 │
├──────────┼─────────┤
│        3 │      -1 │
╘══════════╧═════════╛

Plissken killed himself.
CLYDE killed himself.
Machete rode McClane's rocket.
Leone rode Ripley's rocket.
CLYDE killed himself.
Machete rode McClane's rocket.
Plissken killed himself.
CLYDE killed himself.
time: 9.00 minutes
╒══════════╤═════════╕
│   Player │   Frags │
╞══════════╪═════════╡
│        2 │     -24 │
├──────────┼─────────┤
│        3 │      -1 │
╘══════════╧═════════╛

CLYDE killed himself.
Leone killed himself.
CLYDE killed himself.
Plissken rode Leone's rocket.
CLYDE killed himself.
McClane killed himself.
CLYDE killed himself.

clyde

sudo ./run.sh clyde
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcuda.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcurand.so locally
Press 'Q' to abort network game synchronization.
Total players: 3

Waiting for other players: ..  [ 2/ 3]
Console player number: 1
Received All Here, sending ACK.
Received All Here, sending ACK.
Received "Go."

Total players: 3

Found ghost (node 1, player 1)
Found CLYDE (node 0, player 2)
Found F1 (node 2, player 3)


I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:924] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
I tensorflow/core/common_runtime/gpu/gpu_init.cc:102] Found device 0 with properties:
name: GeForce GTX 1080
major: 6 minor: 1 memoryClockRate (GHz) 1.8095
pciBusID 0000:01:00.0
Total memory: 7.92GiB
Free memory: 3.60GiB
I tensorflow/core/common_runtime/gpu/gpu_init.cc:126] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_init.cc:136] 0:   Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:806] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0)
Model loaded: models/checkpoint-27461303

f1

bdd@mlbox:~/vizdoom_cig2017$ sudo ./run.sh f1
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcuda.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcurand.so locally
[0620 17:39:19 @bsds500.py:18] WRN Cannot import scipy. BSDS500 dataset won't be available!
[0620 17:39:19 @svhn.py:18] WRN Cannot import scipy. SVHNDigit dataset won't be available!
[0620 17:39:19 @format.py:17] WRN Error in 'import h5py'. HDF5Data won't be available.
[0620 17:39:19 @format.py:25] WRN Error in 'import lmdb'. LMDBData won't be available.
[0620 17:39:19 @prefetch.py:23] WRN Error in 'import zmq'. PrefetchDataZMQ won't be available.
[0620 17:39:19 @remote.py:10] WRN Error in 'import zmq'. remote feature won't be available
[0620 17:39:19 @concurrency.py:27] WRN Cannot import Future in tornado.concurrent. MultiThreadAsyncPredictor won't be available.
[0620 17:39:19 @gymenv.py:12] WRN Cannot import gym. GymEnv won't be available.
[0620 17:39:19 @simulator.py:33] WRN Error in 'import zmq'. RL simulator won't be available.
Press 'Q' to abort network game synchronization.
Total players: 3

Waiting for other players: ... [ 3/ 3]
Console player number: 2
Received All Here, sending ACK.
Received All Here, sending ACK.
Received "Go."

Total players: 3

Found ghost (node 1, player 1)
Found CLYDE (node 2, player 2)
Found F1 (node 0, player 3)


F1 joined the party!
[0620 17:39:22 @model_desc.py:118] WRN _build_graph(self, input_vars, is_training) is deprecated! Use _build_graph(self, input_vars) and get_current_tower_context().is_training instead.
[0620 17:39:22 @_common.py:61] conv0 input: [None, 120, 120, 24]
[0620 17:39:22 @_common.py:69] conv0 output: [None, 60, 60, 32]
[0620 17:39:22 @_common.py:61] conv1 input: [None, 60, 60, 32]
[0620 17:39:22 @_common.py:69] conv1 output: [None, 30, 30, 64]
[0620 17:39:22 @_common.py:61] pool1 input: [None, 30, 30, 64]
[0620 17:39:22 @_common.py:69] pool1 output: [None, 14, 14, 64]
[0620 17:39:22 @_common.py:61] conv3 input: [None, 14, 14, 64]
[0620 17:39:22 @_common.py:69] conv3 output: [None, 14, 14, 128]
[0620 17:39:22 @_common.py:61] pool3 input: [None, 14, 14, 128]
[0620 17:39:22 @_common.py:69] pool3 output: [None, 7, 7, 128]
[0620 17:39:22 @_common.py:61] conv4 input: [None, 7, 7, 128]
[0620 17:39:22 @_common.py:69] conv4 output: [None, 5, 5, 192]
[0620 17:39:22 @_common.py:61] fcimage input: [None, 5, 5, 192]
[0620 17:39:22 @_common.py:69] fcimage output: [None, 1024]
[0620 17:39:22 @_common.py:61] fc-pi-m input: [None, 1074]
[0620 17:39:22 @_common.py:69] fc-pi-m output: [None, 6]
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:924] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
I tensorflow/core/common_runtime/gpu/gpu_init.cc:102] Found device 0 with properties:
name: GeForce GTX 1080
major: 6 minor: 1 memoryClockRate (GHz) 1.8095
pciBusID 0000:01:00.0
Total memory: 7.92GiB
Free memory: 3.49GiB
I tensorflow/core/common_runtime/gpu/gpu_init.cc:126] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_init.cc:136] 0:   Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:806] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0)
[0620 17:39:22 @sessinit.py:69] Restoring checkpoint from model.tfmodel.
[0620 17:39:23 @sessinit.py:131] WRN Variable conv1/b/Adam:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv0/b/Adam:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable SummaryGradient/conv0/b/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv3/W/Adam:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable advantage_beta:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv4/b/Adam_1:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable SummaryGradient/conv1/W/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable SummaryGradient/fcimage/b/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable SummaryGradient/conv0/W/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable normalized_advantage/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable fc-v/b/Adam_1:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable SummaryGradient/conv4/W/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv4/W/Adam_1:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable cost/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable SummaryGradient/conv3/alpha/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable fc-pi-m/b/Adam_1:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv0/b/Adam_1:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv1/alpha/Adam_1:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable fcimage/W/Adam:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable input_deque/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv0/W/Adam_1:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv1/W/Adam:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable entropy_beta/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv4/alpha/Adam:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv4/W/Adam:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable fcimage/W/Adam_1:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv3/W/Adam_1:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable max_predict_reward/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable learning_rate:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable global_step:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable SummaryGradient/fc-pi-m/W/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv3/b/Adam:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable policy_loss/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable SummaryGradient/conv1/alpha/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv3/alpha/Adam:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable xentropy_loss/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv0/W/Adam:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable fc-v/b/Adam:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv1/b/Adam_1:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv3/b/Adam_1:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable explore_factor:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv1/alpha/Adam:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable fc-pi-m/b/Adam:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv3/alpha/Adam_1:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable policy_beta:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable advantage/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable SummaryGradient/fc-v/W/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable beta2_power:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable mean_advantage/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable fc-pi-m/W/Adam_1:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable fc-v/b:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable SummaryGradient/fc-v/b/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable fc-v/W:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable SummaryGradient/conv3/b/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable fc-pi-m/W/Adam:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable SummaryGradient/conv1/b/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable SummaryGradient/fc-pi-m/b/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable fcimage/alpha/Adam_1:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable SummaryGradient/conv4/alpha/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable entropy_beta:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable Max/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable SummaryGradient/conv0/alpha/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv4/alpha/Adam_1:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable beta1_power:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable predict_reward/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv1/W/Adam_1:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable Max/EMA/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv0/alpha/Adam:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv0/alpha/Adam_1:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable SummaryGradient/fcimage/W/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable conv4/b/Adam:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable SummaryGradient/conv4/b/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable SummaryGradient/conv3/W/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable value_loss/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable fcimage/b/Adam:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable fc-v/W/Adam_1:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable fcimage/b/Adam_1:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable SummaryGradient/fcimage/alpha/rms/EMA:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable fcimage/alpha/Adam:0 in checkpoint not found in the graph!
[0620 17:39:23 @sessinit.py:131] WRN Variable fc-v/W/Adam:0 in checkpoint not found in the graph!