howion / activation-functions

Javascript implementation of some activation functions.

activation-functions

Installation

From NPM

$ npm install activation-functions

Available Functions

// x is always considered to be in radians
.Identity(x)
.Inverse(x)
.BinaryStep(x)
.Bipolar(x)
.Logistic(x) | .Sigmoid(x) | .SoftStep(x)
.BipolarSigmoid(x)
.Tanh(x)
.HardTanh(x)
.ArcTan(x)
.ElliotSig(x) | .SoftSign(x)
.Erf(x)
.Sinc(x)
.Sinusoid(x)
.Gaussian(x)
.ISRU(x, a)
.ReLU(x)
.GELU(x)
.PReLU(x, a)
.ELU(x, a)
.SELU(x)
.SoftPlus(x)
.Mish(x)
.SQNL(x)
.BentIdentity(x)
.SiLU(x) | .Swish1(x)

Mish: Official Repsoitory

License

MIT

activation-functions's Issues

Add support for Mish

Would be awesome if you could add Mish to the library.
Find Mish here - https://github.com/digantamisra98/Mish

sinc assigns to argument instead of comparing

Code for sinc goes like ($x = 0) ? 1 : (Math.sin($x) / $x);. It have to be == or ===.

The hyperbolic tanget function adds the terms instead of dividing them

The code of the hyperbolic tanget function is:
return (Math.exp($x) - Math.exp(-$x)) + (Math.exp($x) + Math.exp(-$x));

When, in theory, it should be:
return (Math.exp($x) - Math.exp(-$x)) / (Math.exp($x) + Math.exp(-$x));