eltonsampaio12 / neat-template

Neat-template

The genome class represents a candidate neural network. It keeps track of the weights, biases, and fitness score. The fitness score helps determine the best players of each generation. The higher the fitness score, the better the probability of being chosen as a parent.

export abstract class Genome {
  protected fitnessScore: number;
  protected brain: NeuralNetwork;
}

The player class is where most of the code will be focused. If you were building a game like Flappy Bird, the player class would represent your bird. You can add anything you want to it, like a positioning vector, score, and so on.

The player class extends Genome, so you inherit all its functionalities. Inside the Player, it's mandatory to have an "evaluateFitness" method. But why not just have it implemented inside Genome? With this approach, it's easy to override the method, allowing anyone to have their own fitness function.

From the Player class, you can call the inherited method "predict" from the Genome class. This method receives a list of inputs (player velocity, position, etc.) and returns an output. We use the softmax activation function, which means the outputs will be a probability distribution adding up to 100%. You can interpret it however you want.

import { Genome } from "./genome";
import { NeuralNetwork } from "./neuralNetwork";

export class Player extends Genome {
  private score: number;
  constructor(brain: NeuralNetwork) {
    super(brain);
    this.score = 0;
  }

  getScore(): number {
    return this.score;
  }

  evaluateFitness() {
    this.fitnessScore = this.score * this.score;
  }

  think(): void {
    const outputs = this.predict([Math.random(), 0.3, 0.3]);
    const maxIndex = outputs.indexOf(Math.max(...outputs));
    switch (maxIndex) {
      case 0:
        break;
      case 1:
        break;
      default:
    }
  }
}

Start by import the NEAT class and tensorflow. Set the tensor backend to "cpu", for simple scenaria there no need to use the GPU, and you will get better performance. If you are doing complex stuff, use the GPU.

How to adapt NEAT in your code

import { Neat } from "./neat/neat";
import * as tf from "@tensorflow/tfjs";

tf.setBackend("cpu");

Start by setting the population size, create a new instance of the Neat class, and pass the population size in the constructor.

Set the number of input, hidden and output nodes.

const populationSize = 10;
const neat = new Neat(populationSize);

const inputNodes = 3;
const hiddenNodes = 2;
const outputNodes = 2;

Call the "initializePopulation" method, and pass the input, hidden and outputs nodes.

neat.initializePopulation(inputNodes, hiddenNodes, outputNodes);

Call the evolve method from the NEAT, class. This method will only run when the population gets to zero.

neat.evolveNextGeneration();

The "evolve" method takes care of parent selection, crossover, and mutation. To reset the population, call the "remove" method from NEAT. For example, in the Flappy Bird game, when the player hits a pipe, use the "remove" method from NEAT; it is essential for the algorithm to work.

When removing something from an array, make sure to loop through the array backwards.

const population = neat.getPopulation();
for (let i = population.length - 1; i >= 0; i--) {
  const player = population[i];
  if (player.hit(pipe)) {
    neat.removePlayer(player);
  }
}

Parent Selection

For this implementation, parent selection is done by choosing the two players with the best fitness scores.

Crossover

Crossover is the process of taking the best players of their generation and merging their genes to generate the next one.

For this implementation, in crossover, we take the hidden weights of parent1 and the hidden bias of parent2 for the child's hidden weights and biases. Then we take the output weights of parent2 and the output bias of parent1 for the child's output weights and biases.

For the future, I intend to easily allow anyone to add a different crossover functionality by using the strategy design pattern.

Mutation

For each player of the new generation, we loop through its weights (hidden, output weights, and biases), and there is a probability of it being mutated or not.

if (Math.random() < this.mutationRate) {
  const offset = Math.random() * 2 - 1;
  mutableValues[k][j] = mutableValues[k][j] += offset;
}

You can set the mutation rate by using the Neat class.
neat.setMutationRate(0.3)

Neural Network

For our neural network, we use TensorFlow. In reality, we could implement our own neural network from scratch, as we only need the feedforward part. However, TensorFlow has lots of optimized functionalities that take into account performance. We can also use the GPU for more complex tasks.

Next steps

• Ability to the save the model.
• Ability to load the model.
• Ability to customize the neural network more