anubisthejackle / 2048-deep-learning

2048 AI

Based on 2048 by Gabriele Cirulli -- https://github.com/gabrielecirulli/2048

This AI uses a Deep Learning Artificial Neural Network with reinforcement learning for training to teach itself how to play 2048. At the outset it only knows that there are four possible choices, to swipe up, down, left and right. It learns that it likes to get high scores per swipe. It also likes to have empty squares on the board, and it likes to have high numbers on the board. It also enjoys having a long game time.

I have not currently been able to achieve a 2048 with this AI, but that will come with a proper neural network design, which I am constantly updating.

Headless 2048

I've started working on a Headless clone of 2048, and would love any help you want to provide. http://github.com/anubisthejackle/Headless-2048/

It is designed to run under Node.js and should be as separated as possible to make it work completely on it's own, without being directly tied to any input device or the like. Once that is achieved, I will be reworking the Deep Learning AI to run via Node as well, and use the Headless 2048 to do it's training.

More to come!

2048-deep-learning's Issues

Normalize to Max Value

Right now the network is looking at each value independently. This has worked, but not lead to a win.

Instead, let's make it compare each slot against the maximum value on the board, so it will consistently know what the largest value on the board is, with less guess work.

Remove animation from actuator.

Removing the required animation from the actuator, and making it simply manipulate the internal array, will speed up the training process. It will also be a step in the direction of letting the code move server-side, which will further speed the process up.

Reinforce against distance to 2048

This wouldn't be good in an infinite-play bot, but may help reinforce the desire to get to 2048. Test and figure out.

Max Value on board / 2048

Different Neural Network Algorithm

It's likely that a convolution network isn't sufficient for this type of problem. I'm beginning to think that a better solution would be a Long Short-Term Network. This means using a different neural network platform than ConvNetJS. Synaptic is a quality system that's architecture free, and just so happens to have examples of how to setup a LST Net on their README.md

Weights and Rewards.

I've been pondering weighting and rewards.

I'll update this issue with more research, as I come across it.

Score-based Rewards

Currently the way we offer rewards on score actually stops providing a benefit as time progresses. This is not optimal, but it was the method I chose so I could normalize a variable weight between 0 and 1.

After looking at the problem further, I realized that there actually is a finite range, a mathematical maximum, for the score, and if I were to use that as the maximum, I could then normalize the weight with an ever increasing benefit, instead of how it is now.

• https://www.reddit.com/r/2048/comments/214njx/highest_possible_score_for_2048_warning_math

Shut off Random Move

The training for every move should, in theory, cause the network to never become stagnant. I'm wondering if the random move is causing the network to sporadically fail. Shut this off, and test to see if that improves game longevity.

Allow saving / loading of networks

As the subject says, utilize ConvNetJS' save/load functionality to allow for multi-session networks.

Multiple Bots

This is a larger ticket:

1. Create multiple AIs files
2. Create a drop down beside the Run button that lets you define the bot to run
3. Move run method to inside the AI class, to allow for customization on a per-AI basis.