wayne82 / node-svm

Support Vector Machine for nodejs

node-svm

libsvm (Support Vector Machine library) addon for nodejs

Support Vector Machines

Wikipedia :

Support vector machines are supervised learning models that analyze data and recognize patterns. SVMs belong to a family of generalized linear classifiers and can be interpreted as an extension of the perceptron. They can also be considered a special case of Tikhonov regularization. A special property is that they simultaneously minimize the empirical classification error and maximize the geometric margin; hence they are also known as maximum margin classifiers.

How to use it

First of all, if you are not familiar with SVM, I highly recommend to read this guide.

Here's an example of using it to approximate the XOR function :

var nodesvm = require('node-svm');
var xorProblem = [
  [[0, 0], 0],
  [[0, 1], 1],
  [[1, 0], 1],
  [[1, 1], 0]
];
var svm = new nodesvm.CSVC({ // classification 
  kernelType: nodesvm.KernelTypes.RBF,
  C: 1.0,
  gamma: 0.5
});

svm.once('trained', function(report) {
  // 'report' provides information about svm's accuracy
  [0,1].forEach(function(a){
    [0,1].forEach(function(b){
      var prediction = svm.predict([a, b]); 
      console.log("%d XOR %d => %d", a, b, prediction);
    });
  });
});

svm.train(xorProblem);

Notice :

• There's no reason to use SVM to figure out XOR BTW...
• The example show how to use C-SVC classifier but you can also use :
• NU-SVC with var svm = new nodesvm.NuSVC(options) (classification)
• EPSILON-SVR with var svm = new nodesvm.EpsilonSVR(options) (regression)
• NU-SVR with var svm = new nodesvm.NuSVR(options) (regression)
• ONE-CLASS SVM is not supported for now

More examples are available in the same name folder.

Initialization

Options with default values are listed below :

var nodesvm = require('node-svm');

var svm = new nodesvm.SVM({
  svmType: nodesvm.SvmTypes.C_SVC,
  // kernels parameters
  kernelType: nodesvm.KernelTypes.RBF,  
  degree: [2,3,4],                      // for POLY kernel
  gamma: [0.03125, 0.125, 0.5, 2, 8],   // for POLY, RBF and SIGMOID kernels
  r: [0.125, 0.5, 2, 8],                // for POLY and SIGMOID kernels
  
  // SVM specific parameters
  C: [0.03125, 0.125, 0.5, 2, 8],       // cost for C_SVC, EPSILON_SVR and NU_SVR
  nu: [0.03125, 0.125, 0.5, 0.75, 1],   // for NU_SVC and NU_SVR
  epsilon: [0.03125, 0.125, 0.5, 2, 8], // for EPSILON-SVR

  // training options
  nFold: 4,               // for cross validation 
  normalize: true,        // whether to use mean normalization during data pre-processing
  reduce: true,           // whether to use PCA to reduce dataset dimension during data pre-processing
  retainedVariance: 0.99, // Define the acceptable impact on data integrity (if PCA activated)
  eps: 1e-3,              // stopping criteria 
  cacheSize: 100,         // cache siez in MB        
  probability : false     // whether to train a SVC or SVR model for probability estimates
});

Notice :

• degree, gamma, r, C, nu and epsilon can take one or more values. Example : degree: [2,3,4] and degree: 3 are both corrects
• If at least one parameter as multiple options node-svm will go through all the combinations to see which one gives the best predictions (i.e. maximize f-score for classification and minimize Mean Squared Error for regression).

###Available kernels

• Linear : nodesvm.KernelTypes.LINEAR
• Polynomial : nodesvm.KernelTypes.POLY
• RBF : nodesvm.KernelTypes.RBF
• Sigmoid : nodesvm.KernelTypes.SIGMOID

###Available SVM types

• C_SVC : multi-class classification
• NU_SVC : multi-class classification
• EPSILON_SVR: regression
• NU_SVR : regression

Notice : ONE_CLASS SVM is not supported (yet)

##Training SVMs can be trained using svm#train(dataset, [callback])

Notice : Once trained, you can use svm#getModel() method to backup your svm model. Then you will be able to create new svm instances without having to train them again and again.

Pseudo code :

var svm = new nodesvm.SVM(options);

svm.train(dataset, function(){
  var model = svm.getModel();
  // persist your model...
});

on('something-append', function(){
 // get your model back...
 //...
 // create a new svm
 var newSvm = new nodesvm.SVM({model: model});
 // use it with no new training...
});

##Predictions Once trained, you can use your svm to predict values for given inputs. You can do that :

• Synchronously using svm#predict(inputs)
• Asynchronously using svm#predictAsync(inputs, callback)

If you are working on a classification problem and if you enabled probabilities during initialization (see initialization §), you can also predict probabilities for each class :

• Synchronously using svm#predictProbabilities(inputs).
• Asynchronously using svm#predictProbabilitiesAsync(inputs, callback).

Notice : inputs must be a 1d array of numbers

Features

node-svm provide additional features that allow you to :

• Mean normalize your dataset
• Evaluate your svm against a test file
• Perform cross validation on your dataset
• Evaluate various combinaisons and find the best parameters
• Reduce your dataset dimension using Principal Component Analysis (PCA)

See examples folder for more informations.

How it work

node-svm uses the official libsvm C++ library, version 3.18. For more informations, see also :

• libsvm web site
• Chih-Chung Chang and Chih-Jen Lin, LIBSVM : a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1--27:27, 2011.
• Wikipedia article about SVM
• node addons

Contributions

Feel free to fork and improve/enhance node-svm in any way your want.

If you feel that the community will benefit from your changes, please send a pull request :

• Fork the project.
• Make your feature addition or bug fix.
• Add documentation if necessary.
• Add tests for it. This is important so I don't break it in a future version unintentionally (run grunt or npm test).
• Send a pull request to the develop branch.

#FAQ ###Segmentation fault Q : Node returns 'segmentation fault' error during training. What's going on?

A : Your dataset is empty or its format is incorrect.

###Difference between nu-SVC and C-SVC Q : What is the difference between nu-SVC and C-SVC?

A : Answer here

###Other questions

• Take a look at libsvm's FAQ.
• Create an issue

License

MIT