brainjs / brain.js Goto Github PK

This came to me talking about the neural net when explaining it to a friend, might be fun.

neural-network.js#L171

if (callback && (i % callbackPeriod == 0)) {
    callback({ error: error, iterations: i });
}

Is it possible – send current net state to callback function?

Let me explain.
I divide my training data to real training data and test data.
And I want run nn on test data in some period for look real error.

Code for example what I want:

if (callback && (i % callbackPeriod == 0)) {
    callback({ error: error, iterations: i }, this);
}

I get errors when running npm install:

1. esprima-six is missing
2. canvas does not install

`npm ERR! Linux 3.13.0-37-generic
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "browserify"
npm ERR! node v5.10.1
npm ERR! npm v3.8.3
npm ERR! code E404

npm ERR! 404 Registry returned 404 for GET on https://registry.npmjs.org/esprima-six
npm ERR! 404
npm ERR! 404 'esprima-six' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404 It was specified as a dependency of 'derequire'
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

npm ERR! Please include the following file with any support request:
npm ERR! /var/www/brain.js/npm-debug.log
`

I see a lot of usage of neural nets to do things like play games, or detect cats, or make crazy cool images. I think those are valuable, but what about something that could save someone's life?

I propose a project that we feed xray images, and it can detect cancer.

Feedback welcome.

Likely need to work with npm team to figure this out – but ideally, npm i brain installs this package vs. the old one.

I'm fairly new to implementing neural networks and am using this library for the first time. I'm doing text processing and so far I'm up to 5.6 GB of RAM required on a simple bigram.

The way I'm structuring the training data is like this:
{ input: {[inputWord]: 1}, output: {[outputWord]: 1 } }

so if it were "cat sat" the training data would look like:
{ input: {cat: 1}, output: {sat: 1} }

I have two questions:

1. what's a good way to predict memory requirements? Is there a back-of-napkin big-O?
2. is there a better way to structure the data in this particular case?

Thanks!

I recently cloned the repository and went to run the stream example and got an error that the stream write could not receive a null value. I suspect it's because streams have changed in that last few node iterations.

Any suggestions on a fix? I seems that taking out the write(null) from stream-example fixes the error but then it doesn't fire the statistics of the stream iterations or provide a toJSON output.

Will work on more details when on computer.

Updating now, just thought I'd make an issue for tracking.

fails on multiple accounts:
`> mocha test/cross-validation --timeout 10000

(node) child_process: options.customFds option is deprecated. Use options.stdio instead.

OCR cross-validation
Cross validating
1) recognize characters in different fonts

0 passing (563ms)
1 failing

1. OCR cross-validation recognize characters in different fonts:
   TypeError: Cannot read property 'learningRate' of undefined
   at testPartition (/var/www/brain.js/lib/cross-validate.js:21:28)
   at /var/www/brain.js/lib/cross-validate.js:60:18
   at Array.map (native)
   at crossValidate (/var/www/brain.js/lib/cross-validate.js:55:28)
   at Context. (/var/www/brain.js/test/cross-validation/ocr.js:63:18)
   at callFn (/var/www/brain.js/node_modules/mocha/lib/runnable.js:250:21)
   at Test.Runnable.run (/var/www/brain.js/node_modules/mocha/lib/runnable.js:243:7)
   at Runner.runTest (/var/www/brain.js/node_modules/mocha/lib/runner.js:373:10)
   at /var/www/brain.js/node_modules/mocha/lib/runner.js:451:12
   at next (/var/www/brain.js/node_modules/mocha/lib/runner.js:298:14)
   at /var/www/brain.js/node_modules/mocha/lib/runner.js:308:7
   at next (/var/www/brain.js/node_modules/mocha/lib/runner.js:246:23)
   at Immediate._onImmediate (/var/www/brain.js/node_modules/mocha/lib/runner.js:275:5)
   at tryOnImmediate (timers.js:534:15)
   at processImmediate as _immediateCallback
   `

• when added trainOpts, this is the output err:

`Training iterations per second: 45.87900933820544
1) recognize characters in different fonts

0 passing (3s)
1 failing

1. OCR cross-validation recognize characters in different fonts:
   ReferenceError: assert is not defined
   at Context. (/var/www/brain.js/test/cross-validation/ocr.js:84:5)
   at callFn (/var/www/brain.js/node_modules/mocha/lib/runnable.js:250:21)
   at Test.Runnable.run (/var/www/brain.js/node_modules/mocha/lib/runnable.js:243:7)
   at Runner.runTest (/var/www/brain.js/node_modules/mocha/lib/runner.js:373:10)
   at /var/www/brain.js/node_modules/mocha/lib/runner.js:451:12
   at next (/var/www/brain.js/node_modules/mocha/lib/runner.js:298:14)
   at /var/www/brain.js/node_modules/mocha/lib/runner.js:308:7
   at next (/var/www/brain.js/node_modules/mocha/lib/runner.js:246:23)
   at Immediate._onImmediate (/var/www/brain.js/node_modules/mocha/lib/runner.js:275:5)
   at tryOnImmediate (timers.js:534:15)
   at processImmediate as _immediateCallback
   `

Before doing that I'd recommend:

• having clear installation steps
• improving documentation
• have clear examples

In readme.md:

By default brain.js uses one hidden layer with size proportionate to the size of the input array.

This mean like count input data dimension equal to one hidden layer.
Mm.. sorry for my english. I talk for example, if input array dimension to 100 than and hidden layer will contain 100 neurons.

But in sources of function train() write:
sizes.push(Math.max(3, Math.floor(inputSize / 2)));

What right?

And what you recommend about create custom sizes of hidden layers?

An idea in collaboration with @ddsol.

Would be fantastic to have a nice visualization of a network. Here is a really nice example: http://bl.ocks.org/mbostock/4062045

Hi!
How can I read readme.md, I need normalize input data to range 0..1.
That's ok with linear data, same as color, price, year, etc.

But look at my example:
Form in my web page have one field with radio button, like:

• credit;
• debet;
• personal card;
• other.

How I need provide this data for train nn?
Ok, I can send something like:

• type__credit: 0;
• type__debet: 0;
• type__personal_card: 1;
• type__other: 0.

But this make more input columns, even for many fields/values, as i showed.
Then I thought, what will be if I will make this:

• for «credit»: type=0.25;
• for «debet»: type=0.5;
• for «personal card»: type=0.75;
• for «other»: type=1.

That's save me from many columns, but will be work?

And what I can do for checkboxes? Maybe:

var typeForInputBrain = 0;

typeForInputBrain += (typeFromForm.indexOf('credit')) ? 0.04 : 0;
typeForInputBrain += (typeFromForm.indexOf('debet')) ? 0.08 : 0;
typeForInputBrain += (typeFromForm.indexOf('personal card')) ? 0.16 : 0;
typeForInputBrain += (typeFromForm.indexOf('other')) ? 0.32 : 0;

Then even if all values was checked input value save in range between 0 to 1.
Ok? Is it working method, or, that's way require something new activation function?

P.S. Sorry for my english :)

This is an issue raised by @nickpoorman.

I've been trying to think of a concept where I can make the likely method not attached to the neural network. Currently it is:

var brain  = require('brain')
  , net    = new brain.NeuralNetwork
  ;
net.train([]);

var result = brain.likely([]);

Are you suggesting something like:

var brain  = require('brain')
  , likely = require('lib/likely')
  , net    = new brain.NeuralNetwork
  ;
net.train([]);

var result = likely(net, []);

?

As this projects grows (again), I guess it would make sense to have an organization for it.

Things like brain-examples, brain-visualizations could be stored there as well, in separate repos.

not predict(input) -> input + output

I'd like to update the syntax to that of es6, thoughts?

In the past I have used Brain.js and some other JavaScript based machine learning libraries but unfortunately I have found them not matching my needs. That's why, working on my personal projects, I have developed the idea of creating a different approach by myself. So, sorry to be here to talk about another project but I would really like to receive some opinions and suggestions from experienced people that, being into this specific field, could help me to set useful and shared expectations. By the way, thanks for the great support given to Brain.js so far. The library is called DN2A and is on https://github.com/dn2a/dn2a-javascript

For debian based Linux distribution, canvas dependency installation requires install of cairo package

Actually, why is this required?

Any chance of extending this service to support time series predictions?

It'd be really cool to have something like this: http://karpathy.github.io/2015/05/21/rnn-effectiveness/ in brain.js

var net = new brain.default.NeuralNetwork();

var iter = 0;
walk.walkSync('./data', function(basedir, filename, stat) {
	var trainingData = require('./data/'+filename);
	
	var trainResult = net.train(
		trainingData, {
			initialization: (iter === 0),
			keepNetworkIntact: (iter > 0)
		}
	);
	
	iter++;
});

I expect that this may train nn on different files with trainig data, but in result nn I had nn what trained for first file with trainig data. As if all new runs train not saved, but busy CPU and time.

What I need to do for this?

P.S. I do fs walk because trainig data very large - 2GB.

Investigation started here, but we needed an issue to track for merging into master at some hopeful date. The work will be done here until it is (hopfully) merged.

The standard network uses:

[
  {input: { r: 0.03, g: 0.7, b: 0.5 }, output: { black: 1 }},
  {input: { r: 0.16, g: 0.09, b: 0.2 }, output: { white: 1 }},
  {input: { r: 0.5, g: 0.5, b: 1.0 }, output: { white: 1 }}
]

for training. Would it be possible to get something like that into the recurrent neural network? If so, how?

Hey
Would like to know if the inputs objects support text strings.

I did something like this

var brain = require('brain.js')
var net = new brain.NeuralNetwork();

net.train([
           {input: "my unit-tests failed.", output: "software"},
           {input: "tried the program, but it was buggy.", output: "software"},
           {input: "i need a new power supply.", output: "hardware"},
           {input: "the drive has a 2TB capacity.", output: "hardware"}
         ]);

It outputs { error: NaN, iterations: 1 }

One of the comments says that rnn supports them, so is it only available for it or other models can also make use of strings?

NPM has a different version? 0.7.0

New release?

Is there a way to train the brain and generate possible results instead of letting the brain analyse a certain input?

For example:
"Give me 4 colors which match 'orange' with at least 80% accuracy".

As pseudo-code:

var net = new NeuralNetwork(); //Create neural network

net.train([{input: {r:1, g:0.65, b:0},  output: {orange: 1}}, //This is orange
           {input: {r:0, g:0.54, b:0},  output: {green: 1}}, //This is green
           {input: {r:0.6, g:1, b:0.5}, output: {green: 1}}, //This is also green
           {input: {r:0.67, g:0, b:1},  output: {purple: 1}}]); //This is purple

var output = net.run({"orange": ">0.8", "results": 4}); //return 4 colors which match 'orange' with at least 80% accuracy

Pseudo-output:

[{r:1,    g:0.65, b:0},
 {r:0.98, g:0.55, b:0},
 {r:1,    g:0.55, b:0.2},
 {r:0.85, g:0.55, b:0}]

A real world example would be:
"Tell me the first thing which comes to your mind when I say Internet". You might say something like Tim Berners-Lee or Wikipedia.

Is something like that possible with BrainJS?
Note: I also posted this question on StackOverflow.

recurrence for the most part

Hey,

first of, cool work :-)! As far as I see it brain.js is currently using a sigmoid/logistic activation function. I think it would be quite beneficial to.

1. Document that this one is used
2. Create different ones

I could implement some of them like tanH which are working in the range of (0,1) which is currently used as output.

Currently this works in memory. Would it be a good idea to extend this to work with any store (e.g. a database)?

That means anyone can build a custom store object and expose the methods to get and set data.

Hi,

Not an issue per say.
It would be very nice to have a simple RNN or so to predict mouse motion.
Did someone see/did such example in a browser?
I have seen such example in c (don't have the reference now).

I don't look for a fastest or smartest example. Just one that work ;-)

Thanks.

Any chance we could get built in data normalization?

var net = new brain.NeuralNetwork();

var trainingSet = [
  {
    input: [0.1, 0.1],
    output: [0.2]
  },
  {
    input: [0.2, 0.2],
    output: [0.4]
  },
  {
    input: [0.5, 0.1],
    output: [0.6]
  },
  {
    input: [0, 0.1],
    output: [0.1]
  },
  {
    input: [0.3, 0.1],
    output: [0.4]
  }
];

console.log(net.train(trainingSet, { // Object {error: 0.000099992701244754, iterations: 15197}
  errorThresh: 0.0001,
  learningRate: 0.3
}));

console.log(net.run([0.2, 0.4])); // [0.5480908025594973]
console.log(net.run([0.1, 0.4])); // [0.45124782633077687]
console.log(net.run([0.3, 0.4])); // [0.6239448854007088]
console.log(net.run([0.5, 0.4])); // [0.721436435561435]
console.log(net.run([0, 0.4])); // [0.3393225809755128]

I've tried a few different approaches and all sorts of input/outputs (and another library actually).

I really can't get accurate data out. Any tips?

We want to eventually run full on gpu, which would be best keeping that in mind?

Would also be nice while doing this to consider where we absolutely need third party plugins. They are used occasionally, but nothing seemingly to complex for native js to handle.

We could add a section with showcases to the readme to provide usage examples and inspirations.
Due to the response of the community I think there might plenty of better-written projects than mine.

I wrote a classifier module that is able to perform multilabel classification on new oservations. It has to be trained with the target-classes alone.
https://github.com/FranzSkuffka/flextractor/tree/master/lib/classifier

What are your thoughts? What did you build already?

First of all, so excited to see someone doing this! Having maintained several open source projects myself, I completely understand why harthur would decide to move on from spending her time maintaining these. But her projects are so useful that they shouldn't be left to languish. Very happy to see the community picking up the torch!

One suggestion I have (if you haven't done it already) is to merge in all the PRs that were submitted to harthur's original brain repo. It should be pretty doable with just a little bit of git fu. They wouldn't show up as PRs that were merged in, just external branches that were added to master, but I think that's ok.

Or another alternative, you could also just comment on all the old PRs and ask the original authors to resubmit them to the harthur-org repo.

Either way, thanks for picking up the torch here! Glad to see actively supported machine learning in JavaScript :)

http://harthur-org.github.io/brain.js/