lankudot / mlgame

A platform for applying machine learning algorithm to play pixel games

MLGame

A platform for applying machine learning algorithm to play pixel games

MLGame separates the machine learning part from the game core, which makes users easily apply codes to play the game. (Support non-python script as the client. Check here for more information.)

For the concept and the API of the MLGame, visit the wiki page of this repo (written in Traditional Chinese).

Requirements

• Python 3.6+
• pygame==1.9.6+
  • pygame==2.0.0 if installs on mac
• Other machine learning libraries you needed

Usage

$ python MLGame.py [options] <game> [game_params]

• game: The name of the game to be started. Use -l flag to list available games.
• game_params: The additional parameters for the game. Use python MLGame.py <game> -h to list game parameters of a game.
  • Note that all arguments after <game> will be collected to this paremeter
• functional options:
  • --version: Show the version number
  • -h: Show the help message
  • -l: List available games
• game execution options:
  • -f FPS: Specify the updating frequency of the game
  • -m: Play the game in the manual mode (as a normal game)
  • -1: Quit the game when the game is over or is passed. Otherwise, the game will restart automatically.
  • -r: Pickle the game progress (a list of "SceneInfo") to log files.
  • -i SCRIPT [-i SCRIPT ...]: Specify the script used in the machine learning mode. For multiple scripts, use this flag multiple times. The script path starts from games/<game_name>/ml/ direcotry. -i ml_play.py means the file is at games/<game_name>/ml/ml_play.py, and -i foo/ml_play.py means the file is at games/<game_name>/ml/foo/ml_play.py. If the file is in the subdirectory of the ml directory, make sure that the subdirectory has a __init__.py file.

Game execution options must be specified before <game> arguments. Use python MLGame.py -h for more information.

For example:

• List available games:

  $ python MLGame.py -l
  

• List game parameters of the game arkanoid:

  $ python MLGame.py arkanoid -h
  

• Play the game arkanoid level 3 in manual mode on easy difficulty with 45 fps

  $ python MLGame.py -m -f 45 arkanoid EASY 3
  

• Play the game arkanoid level 2 on normal difficulty, record the game progress, and specify the script ml_play_template.py

  $ python MLGame.py -r -i ml_play_template.py arkanoid NORMAL 2
  

Play the Game

In default, the game is executed in the machine learning mode. You could play the game in the manual mode by specifying -m flag.

In the machine learning mode, you have to provide the script to play the game, which is put in the games/<game_name>/ml directory. For example, if there is a file ml_play.py in the games/arkanoid/ml directory, by specifying the -i ml_play.py in the command to use that file to play the game arkanoid.

The games in this repository provide ml_play_template.py in their ml directory, which contains simple code for playing the game. You could duplicate the script and modify it, or use this file for the first time execution.

Read Instruction

The game provides README files for detailed information, such as:

• How to execute and play the game
• The information of game objects
• The format of the scene information and the game command

Here are README of games:

• arkanoid
• pingpong
• snake

MLPlay class

The scripts for playing the game must have a MLPlay class and provide the corresponding functions. Here is a template of the MLPlay class:

class MLPlay:
    def __init__(self, init_arg_1, init_arg_2, ...):
        ...

    def update(self, scene_info):
        ...

    def reset(self):
        ...

• __init__(self, init_arg_1, init_arg_2, ...): The initialization of MLPlay class, such as loading trained module or initializing the member variables
  • init_arg_x: The initial arguments sent from the game.
• update(self, scene_info) -> command or "RESET": Handle the received scene information and generate the game command
  • scene_info: The scene information sent from the game.
  • command: The game command sent back to the game.
  • If the scene_info contains a game over message, return "RESET" instead to make MLGame invoke reset().
• reset(self): Do some reset stuffs for the next game round

Non-python Client Support

MLGame supports that a non-python script runs as a ml client. For the supported programming languages and how to use it, please view the README of the mlgame.crosslang module.

Record Game Progress

If -r flag is specified, the game progress will be recorded into a file, which is saved in games/<game_name>/log/ directory. When a game round is ended, a file <prefix>_<timestamp>.pickle is generated. The prefix of the filename contains the game mode and game parameters, such as ml_EASY_2_2020-09-03_08-05-23.pickle. These log files can be used to train the model.

Format

The dumped game progress is a dictionary. The first key is "record_format_version" which indicates the format version of the record file, and its value is 2 for the current mlgame version. The other keys are the name of ml clients which are defined by the game. Its value is also a dictionary which has two keys - "scene_info" and "command". They sequentially stores the scene information and the command received or sent from that ml client. Note that the last element of "command" is always None, because there is no command to be sent when the game is over.

The game progress will be like:

{
    "record_format_version": 2,
    "ml_1P": {
        "scene_info": [scene_info_0, scene_info_1, ... , scene_info_n-1, scene_info_n],
        "command": [command_0, command_1, ... , command_n-1, None]
    },
    "ml_2P": {
        "scene_info": [scene_info_0, scene_info_1, ... , scene_info_n-1, scene_info_n],
        "command": [command_0, command_1, ... , command_n-1, None]
    },
    "ml_3P": {
        "scene_info": [],
        "command": []
    }
}

If the scene information is not privided for the certain ml client, which the game runs with dynamic ml clients, it's value will be an empty list like "ml_3P" in the above example.

Read Game Progress

You can use pickle.load() to read the game progress from the file.

Here is the example for read the game progress:

import pickle
import random

def print_log():
    with open("path/to/log/file", "rb") as f:
        p = pickle.load(f)

    print("Record format version:", p["record_format_version"])
    for ml_name in p.keys():
        if ml_name == "record_format_version":
            continue

        target_record = p[ml_name]
        random_id = random.randrange(len(target_record["scene_info"]))
        print("Scene information:", target_record["scene_info"][random_id])
        print("Command:", target_record["command"][random_id])

if __name__ == "__main__":
    print_log()

For the non-python client, it may need to write a python script to read the record file and convert the game progess to other format (such as plain text) for the non-python client to read.

Access Trained Data

The ml script needs to load the trained data from external files. It is recommended that put these files in the same directory of the ml script and use absolute path to access them.

For example, there are two files ml_play.py and trained_data.sav in the same ml directory:

from pathlib import Path
import pickle

class MLPlay:
    def __init__(self):
        # Get the absolute path of the directory in where this file is
        dir_path = Path(__file__).parent
        data_file_path = dir_path.joinpath("trained_data.sav")

        with open(data_file_path, "rb") as f:
            data = pickle.load(f)

Change Log

View CHANGELOG.md

mlgame's Issues

pip install pygame on anaconda mac error

If the anaconda python environment is python 3.5, can not install pygame.
Solve: using python 3.6
I'm not sure whether using python 3.6 (the tutorial require python 3.5) will occur any unknown error ?

The game hangs when the exception occurred before `comm.ml_ready()`

def ml_loop():
    comm.ml_ready()
    a = 1/0
    ...

The exception can be passed to the main process.
However, with the following code:

def ml_loop():
    a = 1/0
    comm.ml_ready()
    ...

The game hangs and the exception won't be passed to the main process.

希望能增加指定執行哪個檔案的指令

因為可能會有多個不同方法想測試，希望能在ml資料夾中放置不同檔名的py檔，並透過指令來指定要用哪個來執行，而不用每次有想測試時，還要將檔名改為ml_play.py

Change the indent from tabs to spaces in beta 4.0

As title.

請教生成檔案資訊

之前我執行完 python MLGame.py -f 24 arkanoid 3 -r 後會在MLGame_master的資料夾下看到game_inforxxxxx.pickle這個檔，為什麼我後來log file的時候就看不到了?是因為版本更新後就取消產生這個檔案嗎?還是因為有少打什麼指令呢?

另外我發現game_inforxxx.pickle裡plateform(第二個index)的資訊似乎跟在log資料夾下的pickle檔裡的plateform不太一樣(趨勢大致相同，但不完全符合)，請問是為什麼呢?

感謝您的回覆