haje01 / gym-tictactoe

OpenAI Gym Style Tic-Tac-Toe Environment

gym-tictactoe

OpenAI Gym Style Tic-Tac-Toe Environment.

  |O|X
 -----
  |O| 
 -----
 O|X|X

O's turn.
Enter location[1-9], q for quit:

Requirement

Python >= 3.5

Install

git clone https://github.com/haje01/gym-tictactoe.git
cd gym-tictactoe/
pip install -e .

Try example agents

cd examples/
python human_agent.py
python base_agent.py
python td_agent.py

Temporal Difference Agent Commands

Learn

Usage: td_agent.py learn [OPTIONS]

  Learn and save the model.

Options:
  -p, --episode INTEGER  Episode count.  [default: 17000]
  -e, --epsilon FLOAT    Exploring factor.  [default: 0.08]
  -a, --alpha FLOAT      Step size.  [default: 0.4]
  -f, --save-file TEXT   Save model data as file name.  [default:
						 td_agent.dat]
  --help                 Show this message and exit.

Bench

Usage: td_agent.py bench [OPTIONS]

  Benchmark agent with base agent.

Options:
  -p, --episode INTEGER  Episode count.  [default: 3000]
  -f, --model-file TEXT  Model data file name.  [default: td_agent.dat]
  --help                 Show this message and exit

Grid search

Usage: td_agent.py gridsearch [OPTIONS]

  Grid search hyper-parameters.

Options:
  -q, --quality [high|mid|low]  Grid search quality.  [default: mid]
  -r, --reproduce-test INTEGER  Reproducibility test count.  [default: 3]
  --help                        Show this message and exit.

Play

Usage: td_agent.py play [OPTIONS]

  Play with human.

Options:
  -f, --load-file TEXT  Load file name.  [default: td_agent.dat]
  -n, --show-number     Show location number when play.  [default: False]
  --help                Show this message and exit.

gym-tictactoe's Issues

'TicTacToeEnv' object has no attribute '_seed'

After running set up, I get:

>python td_agent.py play -f best_td_agent.dat
Traceback (most recent call last):
  File "td_agent.py", line 590, in <module>
    cli()
  File "D:\anaconda\lib\site-packages\click\core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "D:\anaconda\lib\site-packages\click\core.py", line 697, in main
    rv = self.invoke(ctx)
  File "D:\anaconda\lib\site-packages\click\core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "D:\anaconda\lib\site-packages\click\core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "D:\anaconda\lib\site-packages\click\core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "td_agent.py", line 280, in play
    _play(load_file, HumanAgent('O'), show_number)
  File "td_agent.py", line 296, in _play
    env = TicTacToeEnv(show_number=show_number)
  File "d:\git\soda-cosmos\gym\gym-tictactoe\gym_tictactoe\env.py", line 96, in __init__
    self._seed()
AttributeError: 'TicTacToeEnv' object has no attribute '_seed'

Observation space? self.observation_space = spaces.Discrete(NUM_LOC)

I think observation space should be total number of possible states ( number of states = 3 * 3 * 3)
spaces.Discrete(27).

Values in best_td_agent

Hi, i was trying to develop a RL agent who can play the Tic Tac Toe game and was wondering how did you get the data in the best_td_agent.dat file. Because, it is used as the optimal policy for the your agent to follow. Did you save them after playing many episodes or is there any other methods. Your answer would be really helpful

Different rewards for O and X?

Sorry if this is a dumb question but here you set different rewards based on whether X wins or O wins:
https://github.com/haje01/gym-tictactoe/blob/master/gym_tictactoe/env.py#L8

Why is that?

If X's reward for winning is -1, wouldn't that encourage an agent that is playing for X to always loose?

Thanks.