ehtnamuh / shologuti Goto Github PK

While training using mlagents-learn in the unity editor, the pause function gets ignored and the game keeps reseting every few steps

Make GutiAgent an Abstract class. Needed for code extensibility and adding other types of agents in the future.

Create observation generator class. Will have instance of board and will serve GutiAgent types with specific observations.
The class should be abstract or an interface as it is intended for extension.

Make Manager class for storing players. Manager class can have exposed variables in the editor to allow designers to easily change what type of players they wish to spawn.
The game manager class can have a reference of the player manager class.
The player manager class will also be useful once a Settings class is created to control the major settings of the game during play.

GameManager class needs to be refactored and separated our into more classes.
Possible Solution: Separate out a Scoreboard class and a RuleBook class to handle the management of scoring and rule checking.

Store Min Max agents evaluations in a list. Will help in taking more random actions.
Edit the MinMax function in the MinMaxAI class
If a new state is encountered with the same as the max value, push that move in a MaxValue List
If then, a state is encountered that has a higher value then the previous known max value,
Empty the MaxValue List and then push the newly found move

Possible Solution
Force calling class to supply RuleBook Class with an instance of the board where it wants to test the rule.

Possible solution:
Fetch GutiArray from the board.board class, and use it to calculate a list of indexes that are non traversable using the generated moveList from simulator class.

do it

Possible Solution:
Create simulator class,
The class access to the logical guti board (guti_map) and can map Address objects to their relevant representations
in a 1d array. The move and reverseMove functions can be rewritten to manipulate the 1d array. By directly manipulating the
array instead of creating the array everytime, the performance of simulating the game should improve significantly.

Reward policy needs fine tuning from the Unity side.

AgentMoveFilterClass handles the perspective of the board seen by the Agent and the also converts the moves generation accordingly

Build a Settings page to control environment settings

Do this only after solving Issue#9.
PPOGutiAgent takes same input but instead of a state value
returns vector indicating selected action.

this class may also need a CheckMoveValidity function.

Possible Solution: add an exploration depth variable.
After sending all simulated states to the NN, select the 4 or fewer states with the highest values and expand them
and repeat the initial process of feeding into the NN for every selected state

Player class has too many if else conditions and specialized code for player types
Possible solution:
Create a base Player Class interface
Inherit and create separate
RLAPlayer
MinMaxPlayer
HumanPlayer

Possible Solution:

1. Since the agents are trained from green perspective, when fetching observations from the ObservationGenerator class,
   simply reverse the observation list.
2. besides reversing the observation, the move indexes must also be translated from Green Player perspective to Red Player Perspective everytime.
3. this will lead to a lot of custom tightly coupled code.