aunum / gold

Reinforcement Learning in Go

Overview

Gold is a reinforcement learning library for Go. It provides a set of agents that can be used to solve challenges in various environments. The library further contains a composable tooling for creating agents and visualizing their performance.

Quick Start

go run ./pkg/v1/agent/deepq/experiments/cartpole/main.go

Requirements

• Go >= v13.0
• Docker
• A browser that isn't IE

Agents

All of the agent implementations can be found in pkg/v1/agent each agent has an experiments folder providing demos across various environments.

agent	algorithm
pkg/v1/agent/q	Q Learning
pkg/v1/agent/deepq	Deep Q learning with Double Q
pkg/v1/agent/reinforce	REINFORCE aka Monte Carlo Policy Gradients
pkg/v1/agent/nes	Natural Evolution Strategies
pkg/v1/agent/her	Hindsight Experience Replay
pkg/v1/agent/ppo	Proximal Policy Optimization ⚠️ in progress

Docs

Each package contains a README explaining the usage, also see GoDoc.

Contributing

Please open an MR for any issues or feature requests.

The project is fairly simple to hack on, environments are provided through Sphere, models are provided through Goro.

Feel free to ping @pbarker on Gopher slack.

Roadmap

• More agents, more environments; see Future Thoughts
• Accelerated compute support
• Tuning libraries
• Deployments

Inspiration

• OpenAI Baselines
• The Gorgonia Project
• Gonum
• RL Overview by Lilian Weng
• Open Endedness by Kenneth O. Stanley, Joel Lehman and Lisa Soros
• AI-GAs by Jeff Clune
• The Bitter Lesson by Rich Sutton
• Reinforcement Learning: An Introduction by Rich Sutton and Andrew Barto