petosa / multiplayer-alphazero

PyTorch AlphaZero implementation with multiplayer support [NeurIPS 2019 Deep Reinforcement Learning Workshop]

Journal Article

https://arxiv.org/abs/1910.13012

What is this

An independent implementation of DeepMind's AlphaZero algorithm with support for multiplayer games. AlphaZero is a deep reinforcement learning algorithm which can learn to master a certain class of adversarial games through self-play. Strategies are learned tabula rasa and, with enough time and computation, achieve super-human performance. The canonical AlphaZero algorithm is intended for 2-player games like Chess and Go, though this project supports multiplayer games as well.

Benefits

Clean & Simple

Clear and concise - a no-frills AlphaZero implementation written with Python 3 and PyTorch. Extensively commented and easy to extend. Support for CPU and GPU training, as well as pausing and resuming training.

Modular & Extensible

Easily plug in your own games or neural networks by implementing the Game and Model interface. Only PyTorch models are supported. Example games and networks included with this repo are listed below.

Example Games

• Tic-Tac-Toe
• Tic-Tac-Mo
• Connect 3x3

Example Networks

• SENet

Novel Multiplayer Support

The first of its kind; support for games with more than 2 players.

Tutorial

Training an agent

You can start training your own AlphaZero agents very easily.

python main.py configs/my_run_configuration.json

You must run main.py with the location of a run configuration json file as the first argument.

multiplayer-alphazero's Issues

gym env wrapper

Hello,

would be great to test this. I have implemented a more complex board game:
https://github.com/Counterfeiter/KingdomBuilder

I made a gym env wrapper... but not pushed at the moment. An Issue with your Game wrapper is, that the observation should be get back to game states and find if done is set. In ray rllib alphazero deepcopied the gym env and written back if needed by MCTS.

How could I write a simple gym wrapper for your wrapper?

Thx

Sebastian

License missing

Hello,

I would like to send pull requests and did not see any license file. Could you please add one?

I have added a hidden states variable (e.g. a complete gym env) to the game class. The hash function is also redirected to the game interface, so this could handle games with incomplete information.

I think thats a great improvement to support a lot more games and should gain more interest. This repo is really underestimated compared to other alpha zero implementations.

Thanks

Sebastian