100 / cranium

🤖 A portable, header-only, artificial neural network library written in C99

Cranium is a portable, header-only, feedforward artificial neural network library written in vanilla C99.

It supports fully-connected networks of arbitrary depth and structure, and should be reasonably fast as it uses a matrix-based approach to calculations. It is particularly suitable for low-resource machines or environments in which additional dependencies cannot be installed.

Cranium supports CBLAS integration. Simply uncomment line 7 in matrix.h to enable the BLAS sgemm function for fast matrix multiplication.

Check out the detailed documentation here for information on individual structures and functions.

---

Features

• Activation functions
  • sigmoid
  • ReLU
  • tanh
  • softmax (classification)
  • linear (regression)
• Loss functions
  • Cross-entropy loss (classification)
  • Mean squared error (regression)
• Optimization algorithms
  • Batch Gradient Descent
  • Stochastic Gradient Descent
  • Mini-Batch Stochastic Gradient Descent
• L2 Regularization
• Learning rate annealing
• Simple momentum
• Fan-in weight initialization
• CBLAS support for fast matrix multiplication
• Serializable networks

---

Usage

Since Cranium is header-only, simply copy the src directory into your project, and #include "src/cranium.h" to begin using it.

Its only required compiler dependency is from the <math.h> header, so compile with -lm.

If you are using CBLAS, you will also need to compile with -lcblas and include, via -I, the path to wherever your particular machine's BLAS implementation is. Common ones include OpenBLAS and ATLAS.

It has been tested to work perfectly fine with any level of gcc optimization, so feel free to use them.

---

Example

#include "cranium.h"

/*
This basic example program is the skeleton of a classification problem.
The training data should be in matrix form, where each row is a data point, and
    each column is a feature. 
The training classes should be in matrix form, where the ith row corresponds to
    the ith training example, and each column is a 1 if it is of that class, and
    0 otherwise. Each example may only be of 1 class.
*/

// create training data and target values (data collection not shown)
int rows, features, classes;
float** training;
float** classes;

// create datasets to hold the data
DataSet* trainingData = createDataSet(rows, features, training);
DataSet* trainingClasses = createDataSet(rows, classes, classes);

// create network with 2 input neurons, 1 hidden layer with sigmoid
// activation function and 5 neurons, and 2 output neurons with softmax 
// activation function
srand(time(NULL));
size_t hiddenSize[] = {5};
Activation hiddenActivation[] = {sigmoid};
Network* net = createNetwork(2, 1, hiddenSize, hiddenActivation, 2, softmax);

// train network with cross-entropy loss using Mini-Batch SGD
ParameterSet params;
params.network = net;
params.data = trainingData;
params.classes = trainingClasses;
params.lossFunction = CROSS_ENTROPY_LOSS;
params.batchSize = 20;
params.learningRate = .01;
params.searchTime = 5000;
params.regularizationStrength = .001;
params.momentumFactor = .9;
params.maxIters = 10000;
params.shuffle = 1;
params.verbose = 1;
optimize(params);

// test accuracy of network after training
printf("Accuracy is %f\n", accuracy(net, trainingData, trainingClasses));

// get network's predictions on input data after training
forwardPass(net, trainingData);
int* predictions = predict(net);
free(predictions);

// save network to a file
saveNetwork(net, "network");

// free network and data
destroyNetwork(net);
destroyDataSet(trainingData);
destroyDataSet(trainingClasses);

// load previous network from file
Network* previousNet = readNetwork("network");
destroyNetwork(previousNet);

---

Building and Testing

To run tests, look in the tests folder.

The Makefile has commands to run each batch of unit tests, or all of them at once.

---

Contributing

Feel free to send a pull request if you want to add any features or if you find a bug.

Check the issues tab for some potential things to do.

cranium's Issues

Cranium is awesome

I have rewritten it to work on everything. GPU, Tensorflow, Eigen, PyTorch. You name it, cranium now works on it. Thanks for you neural network.

Completion of error handling

Would you like to add more error handling for return values from functions like the following?

• malloc ⇒ createLayer
• printf ⇒ printMatrix

Add more activation functions

Such as the various ReLU variants

This entails adding the raw functions for them as well as adding entries for them in the serialization/deserialization functions

Incorrect Variable Naming In Sample Application

I think there is an error in the example in the README.md file:

int rows, features, classes;
float** training;
float** classes;

An int variable and a quadratic pointer are both defined with the same name. I think the name of the variable defined in int type and named classes should be cols.

You can apply the fix as follows. You need to edit the other rows affected by this edit as well.

/* In the line below, the "classes" variable has been updated to "cols". */
int rows, features, cols;

Add better learning algorithms (such as Adam)

Fix any documentation issues

Add optional BLAS implementations as a configuration option

Explanation for Training and Classes

This basic example program is the skeleton of a classification problem.
The training data should be in matrix form, where each row is a data point, and
each column is a feature.
The training classes should be in matrix form, where the ith row corresponds to
the ith training example, and each column is a 1 if it is of that class, and
0 otherwise. Each example may only be of 1 class.

This isn't clear to me. Let's say I have a training data set with 8 features
And I want them classified into 4 classes

Training - rows of eight columns one feature per column
1,2,3,4,5,6,7,8
6,2,3,4,7,6,5,4
6,2,3,4,4,5,5,4
3,4,7,6,5,4,3,2
2,3,4,5,6,4,3,3

Classifications - since I have four classes I would think this has 4 columns, each column indicating which class it is in
0,0,1,0
1,0,0,0
1,0,0,0
0,1,0,0
0,0,01
That's five rows and 4 columns. One column per class.

Create network would be:
createNetwork(8, 1, hiddenSizeReg, hiddenActivationsReg, 4, linear);
Network with 8 inputs 1 hidden layer and 4 outputs (1 per class)

But accuracy requires this:
assert(data->cols == network->layers[network->numLayers - 1]->size);

Can you give a better example?

Thanks

Cranium lacks proper prefixing

All functions, macros and datatypes defined by cranium should have a prefix associated with them to avoid having cranium and another library's defined functions conflict during compilation.

For example, you defined MAX in FUNCTION_H, which may conflict with a differently defined MAX from another projects headers. Instead CRANIUM_MAX(...) should be used. You may want to do this with your template guards as well ( CRANIUM_FUNCTION_H ) to avoid a situation where a user would find your header or another library's header mysteriously not being included because both check for and define the same macro definition in their template guards.

MSVC support

Cannot be compiled with MSVC. Fix: variable length arrays declared on stack can be replaced with memory allocation using alloca(), e.g.
Matrix* errori[network->numLayers];
declaration in batchGradientDescent() (optimizer.h) can be replaced by
Matrix** errori = (Matrix**)alloca(network->numLayers * sizeof(Matrix*));

Replace pointer to pointers with pointer and function to get by indices

This will enable BLAS implementations to work with the library

Including cranium in more than one compilation unit results in compile errors

In order to support being used in more than one compilation unit in a C project ( including it in more than one .c file ), you need to apply the term "static" to all functions defined in the headers. I don't see any global data structures, which is good.

For example, given main.c of

#include "cranium/cranium.h"

int main( int argc, char ** argv ){
  return 0;
}

and an other.c of

#include "cranium/cranium.h"

int otherfn( void ){
  return 4 ;
}

you will receive a tirade of errors

$ gcc main.c other.c -lm 2>&1 | head -20
/tmp/ccuIMzd4.o: In function `createMatrix':
other.c:(.text+0x0): multiple definition of `createMatrix'
/tmp/ccpuiGmt.o:main.c:(.text+0x0): first defined here
/tmp/ccuIMzd4.o: In function `createMatrixZeroes':
other.c:(.text+0x72): multiple definition of `createMatrixZeroes'
/tmp/ccpuiGmt.o:main.c:(.text+0x72): first defined here
/tmp/ccuIMzd4.o: In function `createBatches':
other.c:(.text+0x13b): multiple definition of `createBatches'
/tmp/ccpuiGmt.o:main.c:(.text+0x13b): first defined here
/tmp/ccuIMzd4.o: In function `copyValuesInto':
other.c:(.text+0x226): multiple definition of `copyValuesInto'
/tmp/ccpuiGmt.o:main.c:(.text+0x226): first defined here
/tmp/ccuIMzd4.o: In function `printMatrix':
other.c:(.text+0x2e3): multiple definition of `printMatrix'
/tmp/ccpuiGmt.o:main.c:(.text+0x2e3): first defined here
/tmp/ccuIMzd4.o: In function `zeroMatrix':
other.c:(.text+0x382): multiple definition of `zeroMatrix'
/tmp/ccpuiGmt.o:main.c:(.text+0x382): first defined here
/tmp/ccuIMzd4.o: In function `transpose':
other.c:(.text+0x3e4): multiple definition of `transpose'

etc.