stevenmiller888 / mind

A neural network library built in JavaScript

A flexible neural network library for Node.js and the browser. Check out a live demo of a movie recommendation engine built with Mind.

Features

• Vectorized - uses a matrix implementation to process training data
• Configurable - allows you to customize the network topology
• Pluggable - download/upload minds that have already learned

Installation

$ yarn add node-mind

Usage

const Mind = require('node-mind');

/**
 * Letters.
 *
 * - Imagine these # and . represent black and white pixels.
 */

const a = character(
  '.#####.' +
  '#.....#' +
  '#.....#' +
  '#######' +
  '#.....#' +
  '#.....#' +
  '#.....#'
)

const b = character(
  '######.' +
  '#.....#' +
  '#.....#' +
  '######.' +
  '#.....#' +
  '#.....#' +
  '######.'
)

const c = character(
  '#######' +
  '#......' +
  '#......' +
  '#......' +
  '#......' +
  '#......' +
  '#######'
)

/**
 * Learn the letters A through C.
 */

const mind = new Mind({ activator: 'sigmoid' })
  .learn([
    { input: a, output: map('a') },
    { input: b, output: map('b') },
    { input: c, output: map('c') }
  ])

/**
 * Predict the letter C, even with a pixel off.
 */

const result = mind.predict(character(
  '#######' +
  '#......' +
  '#......' +
  '#......' +
  '#......' +
  '##.....' +
  '#######'
))

console.log(result) // ~ 0.5

/**
 * Turn the # into 1s and . into 0s.
 */

function character(string) {
  return string
    .trim()
    .split('')
    .map(integer)

  function integer(symbol) {
    if ('#' === symbol) return 1
    if ('.' === symbol) return 0
  }
}

/**
 * Map letter to a number.
 */

function map(letter) {
  if (letter === 'a') return [ 0.1 ]
  if (letter === 'b') return [ 0.3 ]
  if (letter === 'c') return [ 0.5 ]
  return 0
}

Plugins

Use plugins created by the Mind community to configure pre-trained networks that can go straight to making predictions.

Here's a cool example of the way you could use a hypothetical mind-ocr plugin:

const Mind = require('node-mind')
const ocr = require('mind-ocr')

const mind = Mind()
  .upload(ocr)
  .predict(
    '.#####.' +
    '#.....#' +
    '#.....#' +
    '#######' +
    '#.....#' +
    '#.....#' +
    '#.....#'
  )

To create a plugin, simply call download on your trained mind:

const Mind = require('node-mind')

const mind = Mind()
  .learn([
    { input: [0, 0], output: [ 0 ] },
    { input: [0, 1], output: [ 1 ] },
    { input: [1, 0], output: [ 1 ] },
    { input: [1, 1], output: [ 0 ] }
  ]);

const xor = mind.download()

Here's a list of available plugins:

• xor

API

Mind(options)

Create a new instance of Mind that can learn to make predictions.

The available options are:

• activator: the activation function to use, sigmoid or htan
• learningRate: the speed at which the network will learn
• hiddenUnits: the number of units in the hidden layer/s
• iterations: the number of iterations to run
• hiddenLayers: the number of hidden layers

.learn()

Learn from training data:

mind.learn([
  { input: [0, 0], output: [ 0 ] },
  { input: [0, 1], output: [ 1 ] },
  { input: [1, 0], output: [ 1 ] },
  { input: [1, 1], output: [ 0 ] }
])

.predict()

Make a prediction:

mind.predict([0, 1])

.download()

Download a mind:

const xor = mind.download()

.upload()

Upload a mind:

mind.upload(xor)

.on()

Listen for the 'data' event, which is fired with each iteration:

mind.on('data', (iteration, errors, results) => {
  // ...
})

Releasing / Publishing

CircleCI will handle publishing to npm. To cut a new release, just do:

$ git changelog --tag <version>
$ vim package.json # enter <version>
$ git release <version>

Where <version> follows the semver spec.

Note

If you're interested in learning more, I wrote a blog post on how to build your own neural network:

• How to Build a Neural Network

Also, here are some fantastic libraries you can check out:

• convnetjs
• synaptic
• brain

License

MIT

---

stevenmiller888.github.io  ·  GitHub @stevenmiller888  ·  Twitter @stevenmiller888

mind's Issues

Is that possible to call 'learn' multiple times?

Hi,

Is that somehow possible to learn your network several times?

From all the examples, it looks like you need to pass an array of {input, output} there. However, it would be cool to be able to learn your network several times. So it gets "smarter" and "smarter".

Are there any ways to perform that?

Regards,

todo: easier exporting/importing network weights

todo: add more examples

download() and upload() the Mind

Hi @stevenmiller888 !

Thanks a lot for making this public, amazing work and very helpful for beginners.
I want to use the Mind in predicting the mood of a user based on keystrokes(moment of pressing, not the content).

As I will have lots of data for training, after training the Mind I would like to download it and use it anytime I need to make a new prediction. I tried to use download() and upload() functions. After downloading the Mind, where can I find it? How do I know what will be the parameter for the upload function?

I am new to nodejs and modules and so on.

Thank you!

How can we add multiple output nodes?

For example, provided trainings for letters A, B and C, once we are predicting the results providing image A, it would be great if we could have outputs results as an object looking like:
{
A: 0.99, // this is the correct answer because it has the highest number.
B: 0.01,
C: 0.00
}

At the moment this is not possible, the predict function only returns one value, so it's very limited with what you can do.

Using datasets of length 100+ can return NaN in prediction

Trying with varying datasets of 2 inputs, if the total length of the dataset exceeds 100, with normalized inputs of -1.00 to 1.00, the predict method will tend to return NaN. Default htan is being used. Increasing hidden layers and units seems to help in increasing the total length, but still is this to be expected, I would assume that the neural network should be able to come up with at least some sort of prediction after several iterations.

Division by input in tutorial

Hi Steven, I don't know where else to ask so I'm asking here--

I'm following your useful XOR network tutorial and I'm confused by this step:

Once we get the delta hidden sum, we calculate the change in weights between the input and hidden layer by dividing it with the input data, (1, 1).

What if the input is (1, 0) or (0, 0)? We would be dividing by zero. How should I handle this case?

Later in part 2 you write

Since we’re dealing with matrices, we handle the division by multiplying the delta output sum with the hidden results matrices’ transpose.

But then you're just multiplying the results! No division happens at all. What's going on?

Your provided example doesn't work perfectly.

I've tested several times on your provided example:

let train_data = [
    { input: a, output: map('a') },
    { input: b, output: map('b')},
    { input: c, output: map('c')}
]

mind.learn(train_data)
console.log('From mind: ', mind.predict(character(
    '#######' +
    '#......' +
    '#......' +
    '#......' +
    '#......' +
    '##.....' +
    '#######'
  )))

And each time, I got different results ranging from 0.6 ~ 0.9, which were expected to be 0.5. Here are several of them:

1. From mind:  [ 0.8690720414762252 ]
 2. From mind:  [ 0.9931120996113908 ]
 3. From mind:  [ 0.6954245636611965 ]
 4. From mind:  [ 0.9677281776279311 ]
 5. From mind:  [ -0.2278222080355501 ]
 6. From mind:  [ 0.9054097880591 ]

Was there anything wrong?

The example code always gives ~1

Problem:
Using the Letters -demo code from Readme.md -- it does not give ~0.5 as expected, but ~1 most of the time.

[(http://codepen.io/mikkokam/pen/730368f3f615bc9930d695cc91b80491?editors=1010)]

Mind showing NaN by adding another feature to XOR example

const Mind = require('node-mind');
const mind = new Mind().learn([
{ input: [0, 0, 1], output: [0] },
{ input: [0, 1, 1], output: [1] },
{ input: [1, 0, 1], output: [1] },
{ input: [1, 1, 1], output: [0] },
]);
const result = mind.predict([1, 0]);
console.log(result);

Here's the response that I get:
[ NaN ]

Why its better?

What is the key differences with Brain and why Mid is better?

character is not defined

I want to try the dome in README file but found 'character is not defined' in the statement, where is the method character from?
const a = character( '.#####.' + '#.....#' + '#.....#' + '#######' + '#.....#' + '#.....#' + '#.....#' )

browser script please

This module can work on the browser, right?
Please provide a built script somehere, either on a dist dir in master on a gh-pages branch. Thank you.

Tried to test-drive brain.js but couldn't make it link to the source:
https://jsbin.com/yamivi/edit?html,js

todo: make scaling input data easier

todo: add multiple uploads

If the network's weights were optionally namespaced, then a single mind instance could take multiple uploads.

For example:

var mind = Mind()
  .upload(letters, 'Letters')
  .upload(numbers, 'Numbers');

// Predict the letter 'B' from the network's existing knowledge of letters
var letterResult = mind.predict('Letters', character(
  '######.' +
  '#.....#' +
  '#.....#' +
  '######.' +
  '#.....#' +
  '#.....#' +
  '######.'
));

// Predict the number 4 from the network's existing knowledge of numbers
var numberResult = mind.predict('Numbers', character(
  '#.....#' +
  '#.....#' +
  '#.....#' +
  '#######' +
  '......#' +
  '......#' +
  '......#'
));

todo: allow arbitrary number of hidden layers

your website has been hacked

when I enter mindjs.net I get forwarded to some advertisment stuff

Where is bias in your implementation?

Hi there,

trying to dabble into neural networks with Javascript and stumbled upon your two-part tutorial. Thanks for these very informative posts.
Could you tell me where and how you tackle bias in your code?
Thanks

todo: clean up back propagation

Typescript Support

This is not an issue, but nice to have improvement. Library is great not to mention simple to use, support for types would be awesome.

todo: add more tests

changes for output>hidden always on results[0]?

Hi Steven,

excellent work, thanks.

I am refering to:

https://github.com/stevenmiller888/mind/blob/master/lib/index.js
line 190:
var changes = scalar(multiply(delta, results[0].result.transpose()), learningRate);

This is part of
Mind.prototype.back = function(examples, results) {
and concretely part of...
// output > hidden

I did run your framework with two hidden layers and results do not match to what I get from other sources. I did a bit debugging and found that results[0].result.transpose() in line 190 is always referencing hidden layer 0, which I believe makes no sense in the // output > hidden section, when we have more than 1 hidden layer. However, it might be, that I got something wrong here. Would be nice, if you could have a look into that one.

My best guess was, that results[0].result.transpose() in line 190 needs to be replaced by results[hiddenLayers-1].result.transpose().

Best Regards,
Jersio

can mind take inputs that vary from 0-1? (ex: 0.2, 0.3, 0.5, etc)

can mind take inputs that vary from 0-1? (ex: 0.2, 0.3, 0.5, etc)

like this example from brain.js:

var net = new brain.NeuralNetwork();

net.train([{input: { r: 0.03, g: 0.7, b: 0.5 }, output: { black: 1 }},
{input: { r: 0.16, g: 0.09, b: 0.2 }, output: { white: 1 }},
{input: { r: 0.5, g: 0.5, b: 1.0 }, output: { white: 1 }}]);

var output = net.run({ r: 1, g: 0.4, b: 0 }); // { white: 0.99, black: 0.002 }

...i really like mind.js because of it's simplicity compared with brain.js...but i just couldn't get a proper output..thanks!

NaN returns when first training entry is longer then second

Hi,

My first steps in deep learning don't go so well. I keep getting NaN.
That turned out to be a "lucky" coincidence.

See this full example:

const Mind = require('node-mind');

var asciiToBin = (function () {
    var pad = '00000000';

    return function (str) {
        return str.replace(/./g, function (c) {
            var bin = c.charCodeAt(0).toString(2);
            return pad.substring(bin.length) + bin;
        });
    };
}());

var binToAscii = function (bin) {
    return bin.replace(/[01]{8}/g, function (v) {
        return String.fromCharCode(parseInt(v, 2));
    });
};

function binArray(data) {
  return Array.from(asciiToBin(data)).map(Number);
}

const mind = new Mind()
.learn([
  { input: binArray("the2"), output: [1] },
  { input: binArray("the"), output: [1] }
]);

const result = mind.predict(binArray("the"));

console.log(result);

That returns NaN.
Now if i merely swap the learn inputs around to this:

  { input: binArray("the"), output: [1] },
  { input: binArray("the2"), output: [1] }

Then it magically starts working :)

This is not the only ones where it produces NaN. It does so in the following cases as well (only changing the inputs, not the predict):

  { input: binArray("the "), output: [1] },
  { input: binArray("the "), output: [1] }

Here it seems like the predict has to be at least as long as the shortest input.

  { input: binArray("one"), output: [1] },
  { input: binArray("two"), output: [1] },
  { input: binArray("a"), output: [1] },

Also gives NaN. Puttin "a" on top makes it work (in this case).

This might be a bug, and one that seems rather easy to get? Or i'm doing something else extremely wrong, in which case i'd also love to hear that!

p.s. if one knows a more efficient way to convert a string to a binary array. Or if there is a more elegant way of thing this in this library, please do tell :)

Cheers,
Mark