The aim of this project is to disseminate deep reinforcement learning agents trained by a variety of algorithms, and to enable easy analysis, comparision, and visualization of them. The hope is to reduce friction for further research into understanding reinforcement learning agents. This project makes use of the excellent Lucid neural network visualization library, and integrates with the Dopamine model release.
This software package is accompanied by a binary release of (1) frozen models trained on Atari games by a variety of deep reinforcement learning methods, and (2) cached gameplay experience of those agents in their training environments, which is hosted online.
Dependencies:
- tensorflow (with version >0.8)
- lucid
- matplotlib for some visualiztions
- moviepy (optional for making movies)
- gym (installed with support for Atari; optional for generating new rollouts)
- tensorflow-onnx (optional for exporting to ONNX format)
To install, run setup.py install
after installing dependencies.
import atari_zoo
from atari_zoo import MakeAtariModel
from pylab import *
algo = "a2c"
env = "ZaxxonNoFrameskip-v4"
run_id = 1
tag = "final"
m = MakeAtariModel(algo,env,run_id,tag)()
# get observations, frames, and ram state from a representative rollout
obs = m.get_observations()
frames = m.get_frames()
ram = m.get_ram()
# visualize first layer of convolutional weights
session = atari_zoo.utils.get_session()
m.load_graphdef()
m.import_graph()
conv_weights = m.get_weights(session,0)
atari_zoo.utils.visualize_conv_w(conv_weights)
show()
From the command line you can run: python -m atari_zoo.activation_movie --algo rainbow --environment PongNoFrameskip-v4 --run_id 1 --output ./pong_rainbow1_activation.mp4
Example jupyter notebooks also live in this directory that give further examples of how this library can be used.